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PREFACE 



This Quality Profile for the Schools and Staffing Surveys (SASS) presents and summarizes 
available information about the quality of data from the five surveys that comprise SASS. 
As background, the report also describes the survey design and procedures for each of the 
surveys. It was prepared by Synectics for Management Decisions Inc., a contractor to the 
National Center for Education Statistics, as Task 3 under Contract No. RN-91-0600.01. 

The Quality Profile was written by Thomas B. Jabine, a consultant to Synectics. Tables and 
exhibits were developed by Sue Streett, also a consultant to Synectics. Additional assistance 
from the Synectics staff was provided by Sameena Salvucci and Steven Fink, all working 
under the direction of Wray Smith, Research Director. 

Several people at the National Center for Education Statistics and the Bureau of the Census 
contributed to the development of the Quality Profile. Kerry Gruber was the project 
coordinator for NCES. In addition to reviewing and providing helpful comments on all 
drafts, she and Daniel Kasprzyk assembled relevant source materials and clarified many 
technical issues that arose in the preparation of this report. Other NCES staff who provided 
answers to technical questions and comments on various drafts were Sharon Bobbin, Steven 
Kaufman, Carrol Kindel and Mary Rollefson. 

This main survey data collection and processing operations for SASS are carried out by the 
Bureau of the Census under an interagency agreement, according to specifications developed 
by NCES. Census Bureau staff members who contributed to this report. By providing 
source materials, answering technical questions and reviewing drafts of sections of the report 
were La Terri Bynum, Patrick Healy, Cleo Jenkins and Irwin Schreiner. 

Peer reviewers for the report were Michael P. Cohen, Statistical Standards and Methods 
Division, NCES, Mary Rollefson, Data Development Division, NCES, Karen King, Bureau 
of the Census, Ron Fecso, National Agricultural Statistics Service and Graham Kalton, 
Westat, Inc. Susan Ahmed, Statical Standards and Methods Division, NCES, was the 
adjudicator for the peer review. 

We hope to update this Quality Profile periodically. Comments on the format and content of 
this first version are welcome. 
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CHAPTER 1 



INTRODUCTION AND OVERVIEW 



Every operating system produces information that can tell us 
how to improve it. — George Box (1993) 



1.1 Introduction 

The Schools and Staffing Survey (SASS) is a periodic, integrated system of surveys of 
schools, school districts, school administrators and teachers. SASS is sponsored by the 
National Center for Education Statistics (NCES) of the U.S. Department of Education. Users 
of the survey data are educators, researchers, policymakers and others interested in educational 
issues. The survey data are collected by mail, with telephone followups to nonrespondents. 

Purpose and audience The main purpose of this SASS Quality Profile, which we expect to 
update periodically, is to summarize what is known about the quality of data from the five 
surveys that comprise SASS. As background, we also provide information about the survey 
design and procedures for each of the surveys: the School Survey, the School Administrator 
Survey, the Teacher Demand and Shortage Survey, the Teacher Survey and the Teacher 
Followup Survey. 

This report will be of interest to users of SASS data, to persons responsible for various 
aspects of the design and operation of the five surveys and to anyone interested in the quality 
of survey data, especially data from mail surveys and surveys related to education. More 
specifically, the report will provide the basis for a systematic review of past and ongoing 
research on the quality of SASS data, with a view toward identifying gaps in our knowledge 
and establishing priorities for future research activities. 

Survey rounds We will refer to each repetition of the four basic surveys and the Teacher 
Followup Survey as a round. For each round of SASS, the four basic surveys are conducted 
during a base school yeai and the Teacher Followup Survey during the following school year. 
The timing of the surveys for the first 3 rounds and the planned timing for Round 4 are as 
follows: 

Round Base year Teacher Followup Interval 



2 
3 
4 



1987-88 
1990-91 
1993-94 
1997-98 



1988-89 
1991-92 
1994-95 
1998-99 



n.a. 



3 yrs. 

3 yrs. 

4 yrs. 
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Scope of the report This SASS Quality Profile describes the survey design and procedures for 
each of the SASS surveys for the latest round for which reasonably complete documentation 
was available at the time of writing. Thus, we describe the design and procedures used tor 
the four basic surveys in Round 2, along with information on major changes between Rounds 
1 and 2. For the Teacher Followup Survey, however, only the Round 1 design and 
procedures are described. Information about the quality of SASS data, such as unit and item 
response rates, is presented for all rounds for which it was available when this report was 
being prepared. 

Sources of information about sampling and nonsampling errors Errors in surveys are of two 
kinds: sampling errors and nonsampling errors. Sampling errors are the result of basing 
survey estimates on a sample, rather than all units in the population of interest, Nonsampling 
errors can occur at any stage of a survey, including sample selection, data collection, data 
processing and estimation. 

All publications based on SASS data include information about sampling errors of SASS 
estimates. Each miblication in the Education Data Tabulations (E.D. Tabs) series includes 
separate tables with sampling errors for selected estimates included in the publication. A 
publication summarizing results from Round 1 of SASS (Choy, Henke, Medrich and Bobbin, 
1992) includes a table showing the estimated standard error for each estimate discussed in the 
text of the report and a table showing standard errors for selected public school estimates at 
the state level. Generalized variance functions, which provide approximations of sampling 
errors for all estimates, based on their size, are being developed for each of the surveys for 
both rounds (Salvucci and Holt, 1992; Salvucci, Holt and Moonesinghe, 1994; Salvucci, 
Galfond and Kaufman, 1993.) The documentation given to users of microdata files includes 
formulas and instructions for estimating standard errors of the items included in their analyses. 

Information about nonsampling errors, which are the primary focus of this report, comes from 
several sources: 

Operational or performance data, including unit and item response rates, results 
of supervisory reviews of interviewers' work, results of reinterviews, and pre- 
edit and edit failure rates. 

• Findings from pretests, in-depth group and individual interviews, and 
methodological experiments. 

Micro-evaluation studies, in which the accuracy of a sample of individual 
responses is evaluated by various means, sj-uch as intensive reinterviews o r 
comparison with existing records. 

• Macro-evaluation studies, in which the differences between survey estimates 
and comparable estimates from other sources are analyzed. Such studies may 
involve data from two or more SASS surveys or they may compare SASS data 
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with those from other NCES surveys or from surveys conducted by other 
organizations, such as the Bureau of the Census. 

This report draws on all of these sources. It is based f c .ost entirely on existing 
documentation; with minor exceptions, no new tabulau jns or analyses were undertaken to 
provide new material. We expect, however, that this systematic presentation of existing 
material will make it easier to identify gaps in what is known about the quality of SASS data 
and will lead to efforts to fill those gaps. Information provided by new documentation 
appearing after the beginning of 1994 has not been incorporated, but references to several 
such items, including papers to be presented at the 1994 annual meeting of the American 
Statistical Association and forthcoming contractor reports, have been included at appropriate 
points throughout this report. 

Structure of the report The report has seven chapters. After this introductory chapter, there is 
a separate chapter for each of the five SASS surveys. Each of these chapters has sections 
covering: frame development and sample selection; data collection and associated errors; 
and data processing and estimation (including imputation and estimation of sampling errors). 
Each of the chapters for the 4 basic surveys also includes a section on evaluation of estimates. 
The final chapter discusses the principal sources of error in SASS surveys, reports on the 
status of recent and current research, and evaluation studies, and suggests some ways in which 
data users may take account of sampling and nonsampling errors in SASS data. Exhibits and 
tables appear at the end of each chapter and a list of references follows the final chapter. 

To avoid unnecessary overlap, the description of the sample design and selection for each of 
the five surveys covers only the additional stages of selection for that survey. Thus, for 
example, the sample design and selection procedures for schools are described in Chapter 2, 
The School Survey, and are not repeated in Chapter 3, The School Administrator Survey. For 
unit response rates, survey-specific rates are presented for each survey. In addition, where 
relevant, we present cumulative response rates, taking into account losses at all stages of 
sample selection and data collection. 

Additional sources of information The references cited in this report include several kinds of 
NCES publications. A separate Data File User 's Manual (sometimes referred to as the 
codebook) is available for each of the five Round 1 surveys (NCES, 1991a,b,c,d; Faupel, 
Bobbitt and Friedrichs, 1992) and is provided to purchasers of data tapes from those surveys. 
Common to each of the manuals for the four basic surveys is a section describing the survey 
design and procedures for all four basic surveys. This section is followed by documentation 
of the data files and copies of the questionnaire(s) for the survey covered by that manual. All 
of the contents of the manual for the Round 1 Teacher Followup Survey are specific to that 
survey. For Round 2, a Data File User 's Manual covering all 4 of the basic surveys is now 
available (Gruber, Rohr and Fondelier, 1993). Volume I describes the design and procedures 
for the surveys, Volume II provides documentation of the restricted-use survey data files, and 
Volume III provides documentation of the public-use files. A separate Data File User's 
Manual has also been released for the Round 2 Teacher Followup Survey (Whitener, Rohr, 

• 
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Bynum, Kaufman and King, 1994). 

This SASS Quality Profile is one of a series of technical ana evaluation reports that are "... 
designed for the audience that examines analytical methods, survey design, procedures, or data 
quality issues ..." (Elliott, 1991). Another technical report (Kaufman, 1991) gives a detailed 
description of the sample design and estimation for the four basic surveys in Round 1 and a 
comparable report for Round 2 (Kaufman and Huang, 1993) is now available. 

Seven papers on various aspects of SASS methodology and the quality of SASS data were 
presented at the 1992 annual meeting of the American Statistical Association and are 
published in the 1992 Proceedings of the Section on Survey Research Methods. Two papers 
presented at the 1993 International Conference on Establishment Surveys are included in the 
proceedings of that conference. Additional papers on SASS methodology were presented at 
the 1993 annual meeting of the American Statistical Association and more will be presented at 
the 1994 annual meeting. These papers will appear in the Proceedings of the Section on 
Survey Research Methods for the two years. 

In preparing this report we have relied on published sources whenever possible, but much of 
the information comes from unpublished memoranda and reports. Readers who would like to 
obtain copies of these items or who have questions about SASS findings and methodology 
should write to: 

SASS Quality Profile 

555 New Jersey Avenue, N.W. 

Washington, D.C. 20208-5651 

1.2 An overview of SASS 

Objectives SASS is designed to provide periodic, timely data on public and private 
elementary and secondary schools in the United States. Major categories of data collected 
include school and teacher characteristics, school operations, programs and policies, teacher 
demand and supply, and the opinions and attitudes of teachers and school administrators about 
policies and \yorking conditions. The analytical power of the data is enhanced by the ability 
to link survey data for individual local education agencies (LEAs), schools and teachers. The 
use of comparable questions in each round of SASS makes it possible to monitor changes in 
the nation's elementary and secondary education system. In each round, special inquiries can 
be included, subject to constraints on overall cost and burden on respondents. 

SASS data provide a basis for addressing five major policy issues (Hudson and Darling- 
Hammond, 1987; Office of Educational Research and Improvement, n.d.): 



Teacher supply and demand. In what teaching fields do shortages exist? What 
school characteristics influence teacher supply and demand? How do the 
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characteristics of new hires compare with those of the existing work force? What 
distinguishes teachers who leave the profession from those who stay in it? What 
incentives are used to recruit and retain teachers in areas of shortage? 

• Characteristics of elementary and secondary teachers. How does the training and 
experience of teachers compare for different types of schools? How do teachers' skills 
relate to their fields of assignment? What are the characteristics of specific subgroups 
of teachers, such as bilingual teachers? 

• Teacher workplace conditions. How are teachers affected by working conditions, 
including teaching workloads, student-teacher ratios and resources available for 
teaching and professional development? How do teachers evaluate their working 
conditions? 

• Characteristics of school administrators. What education and experience do the 
nation's elementary and secondary school administrators have? What problems do 
they consider to be serious, and how do they evaluate their influence on school 
policies? 

• School programs and policies. How do schools vary with respect to admission 
requirements, graduation requirements, teacher salaries and benefits, teaching load and 
staffing patterns? How do graduation and college application rates vary by school? 
How many schools have special programs, such as remedial reading and mathematics, 
programs for the handicapped, programs for the gifted and talented, and extended day 
care programs? How many students are served by these programs? 

An overview of the survey design The target populations for the SASS surveys, which are 
defined more fully in succeeding chapters, include U.S. elementary and secondary schools, 
principals and classroom teachers in those schools, former teachers, and local education 
agencies (LEAs) that are responsible for the administration of one or mote public schools. 
For much of the private sector, there is no counterpart to the LEAs: information on teacher 
demand and shortages is collected directly from individual schools. 

A schematic diagram of the sample selection and data collection process for Round 2 is 
shown in Exhibit 1.1. For each round of SASS, sample selection proceeds in stages: 

(1) A sample of schools is selected. The sample is designed to provide separate data for 
public and private schools, with detail by state for the public sector and by association 
group for the private sector. The same sample is used for the School Administrator 
Survey. For the sample of private schools, the questions for the Teacher Demand and 
Shortage Survey are included in the questionnaire for the School Survey. 

(2) Each LEA that administers one or more of the sample schools in the public sector 
becomes part of f Iie sample for the Teacher Demand and Shortage Survey. 
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(3) For each sample school, a list of teachers is obtained and a sample is selected for 
inclusion in the Teacher Survey. 

(4) A subsample of the teachers who participated in the Teacher Survey and continued 
teaching in the same or another school is selected and contacted during the following 
school year for the Teacher Followup Survey. All teachers who responded in the 
Teacher Survey and are no longer teaching in an elementary or secondary school are 
contacted in the Teacher Followup Survey. 

There was a three-year interval between Rounds 1 and 2 and between Rounds 2 and 3. There 
will be a four-year interval between Rounds 3 and 4. As noted in Chapter 7, Section 7.3, the 
question of optimum periodicity for the surveys is currently being reviewed. Data for the 
four basic surveys are collected during a single school year. Most data items refer to that 
school year; questions on enrollment and staffing refer to October 1 of the school year. 
Questions for teachers about their current teaching loads refer to the most recent full week 
that school was in session. 

The purpose of the Teacher Followup Survey, which is fielded approximately one year after 
the Teacher Survey, is to estimate teacher attrition and to provide data about the factors 
related to it. Topics covered include current employment and teaching status, future plans, 
additional training completed or bej. n since the baseline survey and current opinions about 
various aspects of teaching. Teachers who have moved or left the profession are asked about 
their reasons for doing so. 

The main survey operations, including sample selection, data collection and data processing, 
are carried out by the Bureau of the Census under an interagency agreement, according to 
specifications provided by NCES. Questionnaires are distributed by mail, either directly or, 
for the Teacher Survey in Round 1 only, through school coordinates who distributed them to 
the sample teachers in their schools. Completed questionnaires are returned by mail to the 
Census Bureau's clerical processing office in Jeffersonville, Indiana. Telephone followup 
interviews of nonrespondents to the initial questionnaires are conducted by Census Bureau 
field representatives, working from their homes or from Census regional offices, and their 
questionnaires are also sent to the Jeffersonville facility. 

The target populations, sample sizes and other key features of the five SASS surveys are 
summarized in Exhibit 1.2. 

Evolution of the SASS design The first round of SASS integrated the design and operations 
of three existing survey programs: the Teacher Demand and Shortage Surveys, the Public and 
Private School Surveys, and the Teacher Surveys. Prior to Round 1 of SASS, a Public 
School Survey, covering both schools and teachers, had been conducted for school year 1984- 
85 and Private School Surveys had been undertaken in 1983-84 and 1985-86. A Teacher 
Demand and Shortage Survey, covering LEAs and private schools, had been conducted for 
school years 1978-79 and 1983-84 (NCES, 1992). 
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There were significant changes between Rounds 1 and 2 of SASS in the sampling frames 
from which samples of public and private schools were selected. In Round 1, the primary 
frame for each sector was a list of schools purchased from Quality Education Data, Inc. (the 
QED list). For the private sector, the QED list was supplemented with lists obtained from 
several private school associations and by an area sample (see Chapter 2, Section 2.2, for 
details). In Round 2, the main public school frame for SASS was the list of schools 
developed from NCES's 1988-89 Common Core of Data (CCD) program, which includes an 
annual census of LEAs and schools, by state. For private schools, the 1988-89 QED list, 
supplemented by lists obtained from private school associations, had been used as the frame 
for a universe survey, the 1989-90 Private School Survey. The same frame, with information 
added from the Private School Suivey, was used to select the private school list sample for 
Round 2 of SASS. As in Round 1, the list frame was supplemented by an area sample, but 
the number of primary sampling units was increased from 75 to 123 (Gruber, Rohr and 
Fondelier, 1993; Kaufman and Huang, 1993). 

A new feature of the Round 2 sample selection process was a procedure to control the amount 
of overlap between the Round 1 and 2 school samples. The proportion of overlap was varied 
by sector and by stratum within sector, based on an evaluation of the tradeoff between 
improved estimates of change (favoring more overlap) and expected effects on response rates 
(favoring less overlap) (Kaufman and Huang, 1993, Section 4). Additional design changes 
between Rounds I and 2 are described in Chapters 2 to 5, covering the 4 basic surveys. 

The separate School and Teacher Demand and Shortage questionnaires used for private 
schools in Round 1 were combined to form a single Private School questionnaire in Round 2. 
Specific content changes are described in Chapters 2 to 5. 

Round 3 incorporates further changes in content, design and procedures. A student 
questionnaire has become a new component of SASS. Procedures for obtaining student data 
from school records were tested in the spring of 1993 and this method is being used in Round 
3 for students in a subsample of schools; however, direct collection of information from a 
sample of students and their parents is a possibility in future rounds. Round 3 also has a 
library component in which data about library media centers and specialists (librarians) are 
being collected for a subsample of schools. 

The survey questionnaires have been modified to simplify the work of respondents. Some 
questions that required considerable effort but yielded little usable information have been 
dropped. Others have been reformatted for easier completion by respondents. Instructions for 
skipping items not relevant to all respondents have been clarified. New items will provide 

information needed to monitor the National Education Goals for the year 2000, including data 
on topics such as school safety and drug use by students. 

Except for the questionnaires sent I > school districts and private schools, telephone followup 
of nonrespondents in Round 3 will be computer assisted and will be centralized in two 
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locations: Hagerstown, Maryland and Tucson, Arizona. There will also be some small-scale 
testing of the use of an automated data collection instrument for some types of respondents. 
An interactive diskette containing the school questions will be mailed to schools, completed 
by them using their own computer equipment, and returned by mail (Joseph and Oliveto, 
1992). 
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CHAPTER 2 
THE SCHOOL SURVEY 



2.1 Introduction 

The School Survey is the starting point for all five of the SASS surveys. The initial operation 
in each round of SASS is the selection of samples of public and private schools. Samples of 
teachers for the Teacher Survey and the Teacher Follow-up Survey are selected from these 
schools, and data are collected from their principals in the School Administrator Survey. The 
sample for the public school sector in the Teacher Demand and Shortage Survey consists of 
the LEAs associated with the sample of public schools. 

As background for our discussion of the quality of SASS data, we will describe the design 
and procedures for the Round 2 School Survey, which was conducted in school year 1990-91. 
Major changes between Rounds 1 and 2 are also identified. Available information about the 
quality of School Survey data is presented for both Rounds 1 and 2. 

This chapter consists of four sections covering the main phases of the survey operations: 
frame development and sampling (2.2); data collection procedures and associated errors (2.3); 
data processing and estimation (2.4); and evaluation of estimates (2.5). For some phases, 
especially the frame development and sampling, there are substantial differences between the 
public and private sectors, so the design and procedures for the two sectors are described 
separately. 

Separate questionnaires, Forms SASS-3A and 3B, were used in Round 1 for public and 
private schools. Most items were the same on both versions, but there were some additional 
questions for private schools. Each of the questionnaires had three main sections, covering 
school characteristics, staffing patterns and respondent characteristics (primarily the titles and 
identification of person(s) completing the questionnaire). In Round 2 there were three 
separate questionnaires, Forms SASS-3A, 3B and 3C, for public, private and Indian schools, 
respectively. Their contents were similar, but the private and Indian school questionnaires for 
Round 2 accommodated the requirements of the Teacher Demand and Shortage Survey by the 
inclusion of additional items in the section on staffing patterns and a separate section on 
school personnel policies. 

The initial samples of schools for Round 2 were 9,806 public schools, 3,280 private schools 
and 101 Indian schools. Some of these proved to be out of scope and completed 
questionnaires were not obtained for all of those that were in scope. Further details on 
sample sizes and response rates are provided in Tables 2.1 to 2.8. 

2.2 Frame development and sampling 

The target population The target population for the Round 2 School Survey consisted of 
elementary and secondary schools in the United States that were in operation during school 
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year 1990-91. Schools with no students in any of grades 1-12 were excluded, as were schools 
operating only postsecondary education programs. A public school was defined as: 



... an institution that provides educational services for at least one of grades 1 through 
12 (or comparable ungraded levels), has one or more teachers to give instruction, is 
located in one or more buildings, receives public funds as primary support, has an 
assigned administrator, and is operated by an education agency. (Gruber, Rohr and 
Fondelier, 1993) 

Schools in juvenile detention centers, schools associated with publicly operated hospitals and 
schools located on military bases and operated by the Department of Defense were included 
with public schools. 

A private school was defined as "... a school not in the public system that provides instruction 
for any of grades 1-12 where the instruction was not given in a private home" (Gruber, Rohr 
and Fondelier, 1993). In practice, if the question on place of operation (private home versus 
other) was not answered, the responding unit was excluded if it had fewer than 10 students or 
only 1 teacher. 

Schools operated outside the local public school system by Indian tribes, the Bureau of Indian 
Affairs (BIA), or by Indian tribes under contract with the BIA were defined as Indian schools 
and were treated as a separate category for both sample selection and analysis. 

Sample design objectives and considerations Four basic goals guided the sample design of the 
School Survey: 

(1) Provide estimates of acceptable precision for specified domains of analysis. These 
domains included: public schools by state and by level (elementary, secondary and combined) 
within state; total U.S. private schools by association group (see box below); total U.S. private 
schools by level (elementary, secondary and combined); total U.S. public schools with more 
than 25 percent Indian enrollment; and total U.S. Indian schools. The general approach to 
achieving this goal was to select a specified minimum number of schools in each of these 
domains and to allocate the remaining sample schools in a way that would optimize the 
precision of estimates aggregated over domains, such as national estimates for public and 
private schools. 



ASSOCIATION GROUPS. Most private schools are affiliated with a religious body 
(Catholic, Evangelical Lutheran Church in America, Seventh Day Adventist, etc.) or belong to 
an association of schools, such as the National Association of Independent Schools. Some 
schools with religious affiliations are also members of associations. The allocation of the 
private school sample is designed to provide estimates of acceptable precision for each of 
these groups. The number of separately identified groups was increased from 13 in Round 1 
to 18 in Round 2. Listings of the groups used in Rounds 1 and 2 are shown in Tables 2.7a 
and b. 
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(2) Balance the requirements of (he school sample against (he requirements of (he samples of 
LEAs and teachers. An important feature of SASS is the ability of users to link data for 
sample LEAs, schools and teachers for analytical purposes. To make this possible, some 
tradeoffs were required in the sample design. For example, for the public school sector, one 
possibility would have been to start with the selection of a sample of LEAs and then to select 
a sample of schools in those LEAs. However, a simulation study prior to Round 1 (Wright, 
n,d.) showed that using this design, in a configuration that would have provided the target 
sample sizes for both LEAs and schools, would have led to substantial reductions in the 
precision of school estimates as compared with those based on the design actually used, which 
starts with the selection of a sample of schools, Thus, a moderate loss in precision of 
estimates for LEAs has been accepted in order to preserve the precision of estimates for 
schools. 

A similar issue arose in evaluating the relationships of the school and teacher samples, For 
the school sample, the greatest precision for count data would have been achieved by selecting 
schools with equal probability within each stratum or domain. For the sample of teachers, on 
the other hand, an optimum design would probably have been one that selected schools with 
probability proportionate to size (expected number of teachers, based on frame data) and then 
selected teachers within schools at a rate that would make the overall selection probability for 
teachers constant within strata. A compromise solution was adopted, namely, selection of 
schools with probability proportionate to the square root of their (teacher) size and, within 
each stratum, selection of a fixed number of teachers, subject to constraints on the total 
number of teachers selected in a school, 

(3) Minimize overlap between SASS and o(her NCES surveys of schools (Round 1 only). 
During the data collection period for Round 1, the NCES was also collecting data for two 
other sample surveys of elementary and secondary schools: the National Assessment of 
Educational Progress (NAEP) and the National Educational Longitudinal Study of 1988 
(NELS), To minimize the response burden on individual schools, the sample selection 
procedure used for SASS minimized the overlap among the SASS, NAEP and NELS school 
samples while maintaining the initial probability of selection for each school in SASS 
averaged over all possible sch&ol samples for the three surveys, A comparable procedure was 
not used in selecting the sample of schools for Round 2 because the other two surveys were 
not scheduled to be in the field at the same time. 

(4) Control the overlap between the Round 1 and Round 2 samples of schools. Deciding 
how much the Round 1 and 2 school samples should overlap involved a tradeoff between 
anticipated favorable and unfavorable effects of overlap. To provide estimates of change over 
time with maximum precision, the overlap should be as great as possible. However, it was 
thought that response rates for schools being asked to participate a second time might be 
lower than for those that were selected for the first time. The analysis of these tradeoffs is 
complicated by the fact that overlap in schools guarantees overlap in the sample of LEAs 
associated with those schools. 
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A Round 2 pretest conducted early in 1 990 provided some data on response rates for samples 
of overlap and non-overlap schoo It was estimated thai overlap would reduce response 
rates by 5 percent for schools and 1 1 percent for LEAs. To avoid an undue effect on LEA 
response rates, the decision taken was to control the overlap for the public school sample at 
30 percent, which would lead to an expected LEA overlap of 58 percent. For private schools, 
which had a lower overall response rate in Round 1, the decision was to control overlap at 30 
percent for association groups with high response rates (generally 80 percent or more) in 
Round I and to minimize it for the remaining groups (Kaufman and Huang, 1993, part 4). 
This procedure was used only for the list sample of private schools. For those parts of the 
Round 2 area sample which had not been included in the Round 1 area sample, there was no 
overlap. For the areas included in both rounds, the samples for the two rounds were selected 
independently, with no attempt to control overlap. 

The public school sample for Round 2 The primary frame for the Round 2 public school 
sample was the 1988-89 school year Common Core of Data (CCD) file. The CCD Public 
Elementary /Secondary School Universe Survey is an annual census of public schools in which 
NCES obtains a listing of schools, with basic information on characteristics and size, from 
states, A small supplemental frame, not part of the CCD, consisted of a list, obtained from 
the Bureau of Indian Affairs, of tribal schools and schools operated by that agency. 

The Round 2 frame differed from the frame for Round 1, which was a listing of schools 
obtained from Quality Education Data (QED), a private organization. Schools on the QED 
list were defined as physical locations, whereas those on the CCD list were defined as 
administrative units. For example, an elementary school and high school at the same physical 
location but with different principals would have been counted as one school on the QED list 
but were counted as two schools on the CCD list, 

To make it easier to produce estimated school counts under both the QED and CCD 
definitions, the QED school definitions were retained for sampling purposes in Round 2, 
Thus for sample selection purposes the school units in the Round 2 frame were cither CCD 
schools or groups of CCD schools corresponding to a single QED school. However, 
whenever one of these groups of CCD schools was included in the sample, each CCD school 
was considered a separate unit of analysis and data were collected separately for each one. 
Each school in such a group of schools would receive the same sampling weight for 
estimation purposes. The effects of the change in school definitions have been analyzed by 
Kasprzyk, Salvucci, Saba and Zhang (1994), 

The Round 2 public school sample was a stratified sample. The allocation of sample schools 
among the strata was designed to provide estimates of acceptable precision for each of several 
analytical domains. Within each stratum, the schools in the frame were further sorted on 
several geographic and other characteristics. Following the sorting operation, the specified 
number of schools was selected from each stratum systematically with probability 
proportionate to the square root of number of teachers as reported on the CCD file. All 
schools whose measures of size exceeded the sampling interval for the stratum were selected, 
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For the remaining schools, the controls on overlap with the Round 1 sample were built into 
this part of the selection process. 



The private school sample for Round 2 Because of the difficulties of obtaining a complete 
list of private schools, a dual frame approach has been used to select the samples of private 
schools. The list frame for Round 2 was the one that was developed for the 1989-90 Private 
School Survey. It was based on the 1988-89 QED private school list, supplemented by 
schools not included in that source but found on lists supplied by 20 private school 
associations in the spring of 1989. The list frame consisted of 22,600 schools from the QED 
list and 1,586 schools added from association lists. 

To supplement the list frame, an area sample consisting of 123 primary sampling units (PSUs, 
usually counties or groups of contiguous counties) was selected and special efforts were made, 
using classified telephone directories, government offices and other local sources, to locate 
eligible private schools not included in the list frame. At the U.S. level, it is estimated that 
schools identified by these procedures would account for 21 percent of all private schools on 
the list and area frames combined (Kaufman and Huang> 1993, part 3.3). The samples from 
the list and area frames were selected independently. The overall target sample size for 
private schools was 3,270, with 2,670 of these allocated to the list sample and the remaining 
600 to the area sample. 

The list sample was allocated to 216 strata defined by association group, level (elementary, 
secondary, combined) and census region. An initial allocation was made, proportional to the 
estimated number of teachers in each stratum. This allocation was then modified to ensure a 
minimum of 100 sample schools for each association group, except for groups having fewer 
than 100 schools. Within each stratum, schools in the list frame were sorted by state and 
several other variables within state. Following the sorting operation, the specified number of 
schools was selected from each stratum systematically with probability proportionate to the 
square root of number of teachers as reported in the 1989-90 Private School Survey. All 
schools whose measures of size exceeded the sampling interval for the stratum were selected. 
For the remaining schools, the controls on overlap with the Round 1 sample were built into 
this part of the selection process. 

A similar procedure was used to select schools from the area frame for the sample PSUs. 
Within each PSU, schools were stratified by level and then sorted by association group, 
enrollment and alphabetical order of school name. Most of the schools in the sample PSUs 
were selected with certainty. When sampling did occur, the selection probabilities were based 
on the square root of the reported number of teachers. There were no controls on overlap 
with Round I. 

Changes between Rounds 1 and 2 Most features of the school samples, including overall 
sample sizes, were similar in Rounds 1 and 2. The principal changes in frame development 
and sampling procedures for Round 2 were: 
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For public schools, the shift from the use of the QED list to the CCD as the primary 
frame. 

• Reallocation of the public school sample to improve the reliability of estimates of 
elementary and secondary schools for small states and for public schools with high 
Indian enrollment. 

• Inclusion of a sample of Indian schools. 

• Introduction of procedures for controlling the overlap between Round 1 and 2 samples, 
except for the private school area sample. Procedures for minimizing overlap with 
other current NCES surveys of schools were not needed in Round 2. 

• Increase in the number of PSUs for the private school area sample from 75 to 123. 

• Additional efforts, including the 1989-90 Private School Survey, to improve the 
coverage and quality of the private school list frame. 

Evaluation of the sampling frames Two quality-related features of sampling frames are their 
coverage of the target populations for the survey and the accuracy of information which they 
provide for individual units. 

For public schools, it was discovered after the data collection for Round 1 that for the state of 
Nebraska 275 school districts, each with a single elementary school (Class 1 districts) had not 
been included in the school frame developed from the QED list. As a consequence, 
approximately 275 schools, with an average of about 10.2 students per school, were not 
covered by the Round ! School Survey (Hammer and Gerald, 1991, p.22). 

For private schools, the estimated proportions of schools from the list and area frames provide 
an indicator of the completeness of coverage of the list frames. As noted earlier, in Round 2 
about 21 percent of the estimated total number of private schools in the combined frames was 
accounted for by the area sample frame. The corresponding figure for Round 1 was 22 
percent (NCES, 199 la, p.9). Data for Round 2 show that the estimated proportions of schools 
coming from the area frame varied substantially by association group. The area frame 
contribution was especially large for schools that were members of the National Association 
of Private Schools for Exceptional Children (35.4 percent) and the American Montessori 
Schools Society (31.6 percent). At the other end of the spectrum, the area frame accounted 
for fewer than 5 percent of schools associated with the Friends Council on Education, the 
Association of Military Colleges and Schools and Christian Schools International (Kaufman 
and Huang, 1993, Appendix 4, Table 18). 

Another factor that might affect coverage is the lag between the period for which the 
sampling frame was constructed and the reference period for the survey. The public school 
sample for Round 2 of SASS was based on the CCD for school year 1988-89, but the 
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reference period for the School Survey was school year 1 990-9 L Schools beginning 
operation after school year 1988-89 were not covered unless they resulted from a split of an 
existing school or a merger involving one or more existing schools. Now that data from the 
1990-91 CCD are available, it would be possible to identify schools thai were in operation in 
school year 1990-91 but were not ineluded in the sampling frames for Round 2 of SASS, 

Similarly, the private school list sample frame was the one that was developed for the 1989- 
90 Private School Survey, so that schools starting operation after the spring of 1989 would 
normally not be included in the sampling frame. An evaluation of efforts to improve the 
private school sampling frame for Round 2 is given by Jackson and Frazier (1994), 

Some problems occurred in both rounds when the CCD, QED and PSS definitions of schools 
did not correspond precisely with those used in SASS, In some states, administrative 
groupings of schools within local education agencies were listed on the CCD as single 
schools, whereas SASS treats each location within such a grouping as a separate school. In 
California, it was determined in Round 2 that special education programs had been listed on 
the CCD as schools. For example, the Los Angeles special education program had been listed 
as one school, but was found to have a total of 1 15 separate locations, 74 of which were at 
regular schools already included in the CCD, It was necessary, therefore, to obtain lists of 
sites for these multi-site programs, match them against the CCD file, and select samples of 
those that were not in schools already included in the CCD (Kaufman and Huang, 1993, 
Chapter 10), Similar problems were discovered in Illinois and Pennsylvania, 

The private school list frame contained duplicate listings for some schools, usually with slight 
differences In the name or address of the school. Those discovered prior to sample selection 
were removed. Some were discovered after sample selection; these schools received a 
weighting adjustment to account for their increased probability of selection (NCES, 1991a, 
p,8). Occasionally, public schools were found to have been included mistakenly on private 
school lists received from some states. 

In some instances, frames contained incorrect information or lacked information on school 
characteristics that were used in the sample selection process. In Round 1, for example, some 
private schools were reported in the School Survey as being in the Friends, military or 
Christian international association groups, although they had been classified, in the frame, in 
other groups that had been sampled at a much lower rate (all Friends and military schools in 
the list fiame had been selected with certainty). No bias was introduced into the estimates for 
the affected association groups, but their sampling errors were substantially increased 
(Kaufman and Huang, 1993, Appendix 4), 

Current information on number of teachers and enrollment was sometimes lacking for schools 
on both the public and private list frames for Round 2. Because teacher counts were needed 
to determine selection probabilities, they were imputed for these schools. Values were 
imputed from Round 1 of SASS, when available, from the application of assumed student- 
teacher ratios to enrollment figures, or by using the median value for other schools in the 

2.7 



ERLC 



29 



same stratum. One would normally expect a small increase in sampling errors tc result from 
the use of such imputed values in place of reported values. 

2.3 Data collection procedures and associated errors 

In this section we first describe the data collection procedures in Round 2 and the associated 
supervision and quality assurance procedures. We then provide detailed information on 
nonresponse rates, followed by a discussion of measurement errors, based on reinterviews, 
cognitive interviews and other sources. 

Data collection procedures for Round 2 For public schools, the initial mailing of 
questionnaires was preceded by an advance mailing to all LEAs with one or more sample 
schools, providing general information about SASS and asking for their cooperation. There 
was also an advance mailing to schools, public and private, containing similar information 
about SASS and asking them to submit lists of teachers for use in sample selection. 

Initial mailing of the public and private School Survey questionnaires for Round 2 of SASS 
took place in December 1990 and January 1991, somewhat earlier in the school year than in 
Round 1 . The questionnaires were addressed to school principals, who were asked to 
complete and return them to the Census Bureau's Jeffersonville processing office within 3 
weeks. Response to the survey was voluntary. There were no restrictions on who should 
complete the questionnaire; principals who wished to do so could assign someone on their 
staff to complete the questionnaire. 

After 4 to 5 weeks, a second questionnaire was mailed to schools that had not responded. For 
schools that did not respond to the second mailing after about 3 weeks, Census Bureau field 
representatives, working from the Census regional offices in 2 regions and from their homes 
in the other regions, attempted to complete the questionnaires by telephone. They were 
instructed to try to reach school principals during normal working hours, 8:00 a.m. to 5:00 
p.m. Because some of the questionnaire items might require respondents to check school 
records, Census field representatives were expected to offer to hold the line a few minutes, 
call back or accept collect calls from respondents (Bureau of the Census, 1991b). 

Time required for completion of questionnaire The Round 2 questionnaires for private 
schools included a final question "Not counting interruptions, how long did it take to complete 
this survey?" For questionnaires completed and returned by mail, this item was answered by 
the school employee who completed the questionnaire; for questionnaires completed in 
followup telephone interviews, the item was answered by the interviewer. The median time 
for completion was 60 minutes, with an interquartile range of 50 minutes. For about 90 
percent of all schools, the questionnaire was completed in less than 2 hours and 10 minutes 
and for 1 percent it took more than 5 hours. 

Supervision and quality assurance Field representatives who conducted telephone interviews 
with mail nonrespondents mailed their completed questionnaires to their regional offices on a 
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flow basis. For each field representative, the first 2 questionnaires received were reviewed for 
errors by regional office staff. If a total of 10 or more errors was found in the 2 
questionnaires, the field representative was to be notified of the errors and given suggestions 
for improvement. This process was repeated for successive sets of 2 questionnaires until the 
field representative succeeded in completing a set with fewer than 10 errors (Gruber, Rohr 
and Fondelier, 1993). Some findings from these reviews are reported below under 
"Measurement error, findings from other sources". 

In both Rounds 1 and 2 of SASS, reinterviews were undertaken for samples of completed 
questionnaires for all surveys except the Survey of Teacher Demand and Shortage for LEAs. 
The purpose of the reinterview program in the first 2 rounds was to estimate components of 
error, such as the simple response variance, a measure of the inconsistency of responses over 
repeated applications of a question. In the reinterviews, respondents are asked to answer a 
subset of the questions to which they responded initially. 

Reinterviews for Round 1 were conducted by telephone for all surveys. The Round 1 
reinterviews covered selected questions from both the School Survey and the School 
Administrator Suivey on a single questionnaire. For the Round 2 School Survey, matched 
mode reinterviews were undertaken, that is, schools returning their questionnaires by mail 
were reinterviewed by mail (with telephone followups as necessary) and those responding by 
telephone were reinterviewed by telephone. A sample of 1 123 schools was selected for 
reinterview, nearly 10 percent of the initial sample. Overall, reinterviews were completed for 
91 percent of the schools in the reinterview sample that had responded to the initial 
questionnaire. A few of the reinterviews for schools that initially responded by mail had to 
be completed by telephone, but the great majority of reinterviews used the same mode as the 
initial response. Results from the School Survey reinterviews are presented below, under the 
heading "Measurement error, findings from reinterviews". 

Mode effects There have been no controlled experiments to compare the quality of mail and 
telephone response in SASS. Parmer, Shen and Tan (1992) reviewed information relevant to 
mode effects in the Round 2 School Survey. As shown in Tables 2.1 to 2.4, mail response 
rates (mail responses as a percent of mail plus telephone) varied substantially amoi.g different 
subgroups of the school universe. Table 2.1 shows that about two-thirds of the public schools 
responded by mail, but only 56 percent of the private schools (the data for private schools 
were based only on the list frame sample schools). Public school mail response rates (Table 
2.3) varied widely by state, from 47.9 percent for the District of Columbia to 81.1 percent for 
Delaware. For the private schools, mail response rates varied widely by association group 
(Table 2.4), from 30.7 percent for the American Association of Christian Schools to 73.6 
percent for the Lutheran, Missouri Synod schools (based on the private school list sample 
only). 

Because it was not a controlled experiment, the study did not provide any conclusive evidence 
about mode effects. There was some evidence of differences for private schools, but the 
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analysis procedure used did not provide any indication of which mode produced more 
accurate information. Item nonresponse rates were found to be higher for the mail responses. 

Reinterviews, the results of which are discussed in detail below under "Measurement error: 
findings from reinterviews," provide some evidence on mode effects. In the Round 1 
reinterviews, all of which were conducted by telephone, counts of students served by special 
programs, such as bilingual education, showed more evidence of "heaping" in multiples of 
100 than was found for the initial interviews. This finding suggests a hypothesis that 
telephone respondents are less likely to refer to records or to arrive at a carefully considered 
estimate than those who respond by mail (Bushery, Royce and Kasprzyk, 1992). 

Chances in Round 3 and beyond A pretest for Round 3 was undertaken during the 1991-92 
school year. One new feature was the mailing of a reminder postcard to all schools two 
weeks after the initial mailing to let them know that another form would be mailed if they did 
not mail the first one back. The primary goal of this procedure was to increase the proportion 
of questionnaires returned by mail and reduce the number of more costly telephone follow 
ups. 

In 1992, Census Bureau subject matter and programming staff started work on the 
development of a prototype automated data collection instrument for use in one or more of 
the SASS surveys. Schools or other units willing to use this mode would receive a diskette 
containing the survey questionnaire and instructions for completing it. Using their own 
microcomputers, they would enter their responses on the diskette and return it to the Census 
Bureau. There will be some small-scale testing of the prototype in Round 3. If the test 
results are encouraging, this mode of data collection may be made available to respondents for 
some surveys in subsequent rounds. 

Nonresponse error The two types of nonresponse in the School Survey are unit nonresponse, 
in which no questionnaire of acceptable quality is obtained from an eligible school and item 
nonresponse in which entries are missing for one or more items on a questionnaire. At this 
time, there is no direct evidence or the magnitude of biases caused by these two kinds of 
nonresponse. However, information about the levels of unit nonresponse for different 
subgroups of the survey population and the levels of item nonresponse for different 
questionnaire item., provides some indication of the potential effects of nonresponse bias 
(Moonesinghe, Smith and Gruber, 1993; Scheuren, Parke and Bureika, 1994). 

Unit nonresponse may occur at various points in the data collection process. For public 
schools, a few LEAs in both rounds of SASS have refused participation completely, that is, 
they declined to complete the LEA questionnaire for the Teacher Demand and Shortage 
Survey and they specifically requested NCES not to ask schools in their district to participate. 
In Round 1, 35 school districts with 63 sample schools initially refused to have their schools 
participate in SASS. After contacts by Census Bureau representatives, 17 of these districts 
with 24 sample schools reconsidered their positions and agreed to allow the Census Bureau to 
mail questionnaires to individual schools in their districts (Nash, 1988). Thus, the ultimate 
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loss of schools at this stage was less than 0.5 percent of the public school sample. Although 
only a few schools and school districts were lost to the Round 1 and 2 surveys at this stage of 
data collection, a few large districts were lost in each round, with adverse consequences for 
the quality of data for the states in which those districts were located. 

Most of the unit nonresponse was associated with individual schools. Table 2.5 shows 
unweighted and weighted response rates for Rounds 1 and 2, for public and private schools. 
Tables 2,6 and 2.7 show weighted response rates for public schools by state and private 
schools by association group. Weighted response rates take into account the probabilities with 
which schools in various strata were selected and are therefore a better indicator of the effects 
of nonresponse on survey estimates. The base for each of the response rates shown is the 
number of sample schools that were found to be eligible for the survey. Schools that were 
not operating in the school year of reference for the survey or that failed to meet the 
definition for other reasons were excluded. Table 2.8 shows that 4.0 percent of public 
schools, 5.6 percent of private schools and 1.0 percent of Indian schools were excluded for 
such reasons in Round 2. 

As shown in Table 2.5, response rates for public schools were substantially higher than those 
for private schools in both rounds. Response rates for both sectors were higher in Round 2, 
with an increase over Round 1 of 3 percentage points for public schools and 5 percentage 
points for private schools (based on weighted rates). This result was encouraging in the light 
of Round 2 pretest results which had suggested that response rates might be lower for schools 
that had been in the sample in Round I , However, it may have been due in part to more 
lenient criteria used in Round 2 for the amount of item nonresponse that could be present in a 
questionnaire before classifying it as not acceptable. 

Within each sector there was substantial variation. In Round 1, in the public sector, 24 states 
had weighted response rates of 95 percent or better and 3 states were below 80 percent (Table 
2.6). In Round 2, there were 34 states with rates of 95 percent or better and none below 80 
percent, the lowest rate being 81,0 percent for Maryland. There was wider variation among 
association groups in the private sector, with a spread of nearly 40 percentage points in the 
weighted response rates between the highest and lowest group in both rounds (Tables 2.7a and 

b). 

Although, as mentioned in Section 2.2, it had been felt that schools included in the sample for 
a second round might have lower response rates, this did not happen. Response rates in 
Round 2 for overlap and non overlap schools were as follows: for public schools, 95.0 percent 
for overlap schools (those in the sample for both rounds) and 95.1 percent for nonoverlap 
schools, and for private schools, 87.1 percent for overlap ^hools and 84.5 percent for 
nonoverlap schools (Kaufman and Huang, 1993, Table 2). 

Additional analyses of school response rates for Round 2 by Moonesinghe, Smith and Gruber 
(1993) showed that smaller public schools (in terms of enrollment) tended to have higher 
response rates than larger schools and that those with low proportions of minority students 
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enrolled had higher response rates than those with higher proportions of minority students. 
For private schools, the response rate for schools on the list frame (86.6 percent) was 
substantially greater than the rate for schools from the area frame (74.0 percent). 

The available data on item nonresponse are somewhat more difficult to interpret and 
summarize. In general, the rates refer to the status of each item after edits but prior to 
imputation, and the base for the each rate is the number of questionnaires for which the item 
should have been answered. Data from published summaries of unweighted item response 
rates for the School Survey in Rounds 1 and 2 (NCES, 1991a; Gruber, Rohr and Fondelier, 
1993) are shown in Table 2.9. The rates are based on completed questionnaires; they do not 
include questionnaires that were classified as nonresponse cases because of an unacceptably 
high number of unanswered items. 

Item nonresponse rates tended to be higher for items requiring respondents to report numerical 
amounts than for those requiring a choice among two or more categories. A common 
problem in both Rounds 1 and 2 was failure to check boxes for "none" when that was called 
for. Item nonresponse problems in Round 1 led to several changes in the content and format 
of the questionnaires. Because of these and other changes, the results for the two rounds 
shown in Table 2.9 are not directly comparable. Subject to this caveat, they suggest that item 
nonresponse was somewhat less of a problem in Round 2. 

One item that caused particular problems in Round 1, for both public and private schools, was 
an item on staffing patterns that appeared in the form of a 3 x 28 grid (Item 32 on the 
questionnaire for public schools and 35 on the questionnaire for private schools). The 3 
columns asked for: number of teachers as on October 1, 1986; number of those no longer 
teaching on October 1, 1987; and number of teachers in the category on October I, 1987. In 
the 28 rows, teachers were to be classified by 27 different primary fields of assignment, with 
a total in the final row. This item appeared to be difficult for respondents to complete, as 
indicated by a combination of missing and inconsistent entries. NCES decided that the 
quality of data from this item was unacceptable and did not include the data in either its 
public or restricted use microdata files for the School Survey. In the Round 2 School Survey, 
some parts of this item were dropped; other parts were retained but were asked in a different 
format. 

Other items with high item nonresponse rates in Round 1 for both sectors included an item on 
availability of instruction and size of enrollment in "grades 13 and 14" (covering vocational 
and other "post-graduate" secondary education) and an item asking for a breakdown of prior 
year staff roles of teachers who were no longer in the profession in the current year. In 
Round 2, the grades 13 and 14 categories for enrollment were replaced by a single 
"postsecondary" category. The other item was retained but the number of separate response 
categories was substantially reduced. 

In Round 2, items with response rates below 75 percent included those relating to counts of 
part-time staff and to degree of difficulty in filling vacancies in selected categories. Problems 
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with the items on part-^me staff are attributed partly to the format of the item covering full 
and part-time staff and partly to respondents' uncertainty about the definition of "part-time", 
especially in the smaller schools. 

Measurement error: introduction Information about measurement errors in the School Survey 
comes from several sources, including reinterviews, in-depth interviews using cognitive 
research techniques, reviews of completed questionnaires and analyses of errors and 
inconsistencies detected during data processing. Findings from these sources are presented in 
the next 3 subsections. 

Measurement error: findings from reinterviews The first reinterviewing for the School Survey 
occurred in connection with a large-scale pretest for Round 1 of SASS in the early part of 
1987. The pretest, which was carried out in 10 states, included 220 schools. Of these, 98 
were reinterviewed, by telephone, by Census Bureau interviewers. Unlike the subsequent 
reinterviews following Rounds 1 and 2, which merely asked for second responses to the 
selected questions without any attempt to reconcile differences, the pretest reinterviews called 
for in-depth discussions with respondents about how they had arrived at their initial answers, 
what they had included in their counts and what was excluded. Respondents were also asked 
for their recommendations for improving any of the questionnaire items. 

The report of the reinterviews (Nash, n.d.) included several recommendations for 
improvements in specific questionnaire items, for example: 

♦ For an item on special programs, clarify the definition of bilingual education. 

For an item on student enrollment by grade, clarify the treatment of students enrolled 
under Head Start and Chapter 1 programs. 

• For an item asking about present activities of teachers who had left the school after the 
preceding school year, 38 percent of the schools did not have records available. The 
report recommended further review of nonresponse rates for this item to determine 
how useful the results would be. 

Certain other items that were found to be difficult for some schools to report were 
subsequently dropped from the questionnaire. 

As part of the regular data collection cycle, reinterviews have been conducted by Census 
Bureau field representatives for about 10 percent of all School Survey interviews in Rounds 1 
and 2. The Round 1 reinterviews were all conducted by telephone; in Round 2 they were 
completed, insofar as possible, by the same means as the initial interview. 

Bushery, Royce and Kasprzyk (1992) provide a detailed analysis of reinterview results for 
Rounds 1 and 2. Table 2.10 presents response variance measures for 4 School Survey items 
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that were included in the reinterviews in both rounds. There were moderate statistically 
significant reductions between Rounds 1 and 2 in the gross difference rates for 3 of the 4 

The gross difference rate is the percent of respondents whose responses in the original 
interview and the reinterview were different. The index of inconsistency measures the percent 
of total variance for an item that is accounted for by response variance. The L-fold index of 
inconsistency is used for closed response items with more than two response categories: it is 
a weighted average of the simple index over all categories. As a rough rule of thumb, 
response variance is considered to be low when the simple or L-fold index of inconsistency is 
less than 20, moderate when it is between 20 and 50, and high when it is greater than 50, 
For further discussion of these measures of response error, see Groves (1989) and Forsman 
and Schreiner (1991), 



items and in the index of inconsistency for two of them. Changes in reinterview methodology 
may have contributed to some of the reduced response variance. Al! four had indexes of 
inconsistency in the moderate range in Round 2. 

Looking at all of the factual reinterview items (most of them not the same in both rounds) 
from the School Survey shows the following distribution by level of response variance, as 
measured by the index of inconsistency (Newbrough, 1989; Royce, 1992): 

Round Index of inconsistency 

Low Medium High 

10 6 8 

2 17 12 7 

Although not definitive, because the items and reinterview procedures for the two rounds were 
different, these findings suggest that efforts by NCES and the Census Bureau to improve the 
questionsvond instructions for Round 2 may have had some success. 

For Round 2, Bushery et al compared the results for schools in which mail was used for both 
the original interview and the reinterview with results for schools in which the telephone was 
used for both. Table 2,11 shows the results of this comparison for 4 School Survey items. 
For each item, both measures of response variance were significantly lower for the mail/mail 
group. It occurred to the investigators that one possible reason for this might have been that 
some mail respondents had saved copies of their questionnaires and had used the copies to 
prepare their mail reinterview questionnaires. However, a school-by-school comparison of 
individual responses showed that no more than 6 percent were likely to have done this. They 
hypothesized that the lower response variance for the mail interviews may have resulted 
primarily from two factors: 
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• Only respondents who answered the original survey by mail were eligible for 
the mail reinterview. These respondents were likely to be more cooperative 
and answer the questions more carefully in both interviews, 

• Respondents interviewed by mail may take time to look up the answers to 
questions from records or they may go through a more careful, but more 
lengthy, thought process to provide the needed facts. (Bushcry, Royce and 
Kasprzyk, 1992) 

Measurement errors: findings from in-depth interviews We now turn to a review of findings 
from in-depth initial interviews (sometimes called cognitive or think-aloud interviews) that 
have been conducted with school principals to explore their understanding of and ability to 
answer the questions used in the School Survey. Since the questionnaires are designed to be 
self administered by most respondents, the technique used in these interviews is to ask the 
principals to complete the questionnaire themselves with an observer/interviewer present and 
to describe their reactions and thought processes while proceeding through the questionnaire. 
The observer/interviewer may ask probing questions where necessary to clarify the 
respondents' remarks or behavior. Interviews are tape recorded for use in subsequent 
■analysis. 

In the spring of 1990, members of the Census Bureau's Center for Survey Methods Research 
conducted in-depth interviews with 9 public and 6 private school administrators, using the 
School Survey questionnaires from the pretest for Round 2 (Jenkins and DeMaio, 1990). 
These interviews identified several items asking for counts or percentages that were difficult 
for respondents to answer. An item asking for the percent of students enrolled at the start of 
the previous school year who were still enrolled at the end of that year was subsequently 
eliminated from the final version of the public school questionnaire. Two matrix style items 
on the private school questionnaire that asked for information about full-time equivalent 
teaching staff by grade level and subject or specialty proved to be extremely difficult for 
respondents and were eliminated from the final version of that questionnaire. Other findings 
from the in-depth interviews led to changes in format and wording for some items. 

A second set of in-depth interviews was carried out in the winter of 1991-92 with 20 public 
school principals in 5 mid-western states - Oklahoma, North Dakota, South Dakota, Nebraska 
and Iowa - using a condensed version of the questionnaire that had been used in the pretest 
for Round 3. One objective of the interviews was to learn more about questions that had high 
edit failure rates in Round 2; another was to test new questions that had been developed for 
possible inclusion in Round 3. The particular states were chosen because they had had high 
pre-edit failure rates for student and teacher counts in Round 2. Three of the four schools in 
each state had not been in the SASS school sample previously; the fourth was chosen 
because its School Survey questionnaire had been rejected at the pre-edit stage in Round 2 
due to discrepancies between the teacher and student counts and comparable data from the 
Common Core of Data. 
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A detailed account of each interview is available, as well as a paper (Jenkins, 1992b) and a 
memorandum (Jenkins, Ciochetto and Davis, 1992) that summarize the main findings and 
provide an item-by-item description of the problems that respondents had in answering the 
questions. The paper identifies three types of respondent problems and gives two examples of 
each: 

(A) Misunderstanding of concepts 

Example 1 - Definition of the unit (school) to be covered by the questionnaire. As described 
in Sections 2,4 and 2,5 of this chapter, edits during and after the main processing operations 
in both Rounds 1 and 2 of SASS identified several instances in which school questionnaires 
had been completed for a unit other than the one intended. Such errors were frequently 
associated with schools in veiy small districts and distinct school units located in a single 
building or in nearby buildings with common grounds. In the cognitive interviews, several of 
the principals interviewed had difficulty deciding what units to cover in completing the 
questionnaire. Three of the 20 principals actually reported for a unit other than the one 
intended. Some of the errors and uncertainties were associated with respondents' failure to 
give close attention to the label and instructions on the cover page of the questionnaire, failure 
to display the name of the school prominently on the cover page and misleading cues in some 
of the initial questions. The investigators believed that most problems of this kind could be 
eliminated by redesign of the questionnaire, especially the cover page and label 

Example 2 - Fuli-time versus part-time status. When respondents were asked to give the 
numbers of full ana part-time employees in each of several instructional and support services 
categories, they had difficulty in correctly classifying employees who: had jobs which by 
their nature could not be full-time, for example, bus drivers; worked part-time in more than 
one category, but iuH-time at the school; or worked part-time at the samplr school, but full- 
time for the school district. 

(B) Formal considerations 

Example I - "None" boxes and skip instructions. A series of items about limited-English 
proficient (LHP) students was introduced with a blank space in which to record the total 
number of LEP students and a "none" box underneath it with an instruction to skip the 
remaining i'ems on LEP students if there were none. Several respondents entered "0" in the 
answer space, did not notice the none box and were thus led to a frustrating attempt to answer 
several questions that did not apply to them. Failure to use none boxes and follow skip 
instructions correctly is a fairly common problem, especially with self-administered 
questionnaires. 

Example 2 - Item layout. One item asked respondents, for each of several fields of 
instruction, either to check a box to indicate that they had no vacancies in that field or to 
check one of four boxes indicating the level of difficulty they experienced in filling vacancies 
in that field. Some of the respondents were uncertain how to respond for fields that were not 
relevant in their schools. The intent of a sub-item at the end of this item, asking for 
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specification of subfields in the vocational-technical education category, was especially 
unclear to respondents. 

(C) Use of records 

Example 1 - Use of inappropriate records. One item asked for enrollment by grade on 
October 1 of the current school year. Public schools arc required by law to submit such 
information annually to their school district or to their state. It was the intention of the 
questionnaire developers that respondents use the same information to complete this item. 
However some respondents did not rely on their official fail reports, with the result that they 
went through unnecessary work to complete the item and sometimes provided numbers for a 
different date. 

Example 2 - Converting information in records to the desired format. For an item asking for 
a breakdown of enrollment by standard race/ethnic categories, many respondents recorded the 
numbers for the 4 categories other than white, non-Hispanic, using available records. They 
then derived the number for white, non-Hispanic by subtraction from the total enrollment on 
October 1 that had been reported in the preceding item. However, in some instances, the 
records used to obtain the data for the first 4 categories referred to a date other than October 
1 , leading to a number for white, non-Hispanic enrollment that was not correct for either date. 

The above and other findings from the in-depth interviews were given serious consideration in 
the design of the Round 3 questionnaire for public schools. 

Measurement error: findings from other sources Information on both measurement and item 
nonresponse error is available from a 1992 review of post-edit item response rates, pre-edit 
reject rates and edit change tallies from Round 2 of SASS (Jenkins, 1992a), (The nature of 
the pre-edit and edit operations is described in Section 2.4 below.) School and LEA 
questionnaires had been rejected (for inconsistencies, invalid entries and critical missing data) 
much more often at the pre-edit stage than the questionnaires for individuals - school 
administrators and teachers. The rejection rates were 59.4 percent for public schools and 72.4 
percent for private schools. The private school questionnaire also had a relatively high 
proportion of items with post-edit response rates less than 75 percent, and the reviewer 
concluded that, of all the SASS questionnaires, it was the one most in need of improvement, 
although many of the problems observed were similar for public and private schools. 

Specific problems observed for both public and private schools in this review were similar to 
those identified in in-depth interviews. These included: 

• Frequent inconsistencies between a total reported in one item and breakdowns of that 
total reported in subsequent items. This problem was observed for breakdowns of total 
enrollment, enrollment in grades 10 to 12 and total number of teachers. 

Failure to complete the minutes part of the entries for the item on length of school 
day. 
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• Failure to report numbers of students in selected programs or receiving selected 
services. 

• (For public schools only) Enrollment and teacher counts that were substantially higher 
than those reported for the same schools in the CCD file. The School Survey counts 
for enrollment were at least 35 percent higher for about 8 percent of the schools and 
the teacher counts were higher for about 20 percent. (These differences are discussed 
further in Section 2.5 below.) 

We have referred earlier to the regional office reviews of questionnaires completed by the 
Census Bureau field representatives in telephone followups of mail nonrespondents. In Round 
2, the regional offices were asked to send the Forms SASS-23, on which the review findings 
for their field representatives had been recorded, to the Census Bureau office in Suitland, 
Maryland. A review of these forms led to the following conclusions (Pasqualucci, 1991): 

• Most of the regional offices did not complete the review forms correctly and one 
office apparently had not used them. 

• Many of the field representatives had accumulated more than 10 errors on their first 4 
questionnaires. Common errors were disregard of skip patterns and failure to check 
"None" boxes, entering "0" instead. 

• In the item on the public school questionnaire asking for a breakdown of enrollment 
by race/ethnic category, the total for all categories frequently was not equal to the total 
enrollment reported in a prior question. Some field representatives had entered a 
percentage instead of a whole number for each category. 

• On private school questionnaire items requiring decimal entries, e.g., years of 
instruction required for graduation by subject and items relating to full-time equivalent 
staff, some field representatives ailed to record any digits to the right of the 
preprinted decimal points. 

2.4 Data processing and estimation 

Data processing procedures for Round 2 Exhibit 2. 1 shows, for the Round 2 School Survey, 
the sequence of basic processing operations that occurred between the receipt of 
questionnaires in the Census Bureau's processing facility in Jeffersonville, Indiana and the 
production of a clean data file. As the exhibit shows, activities with large clerical e!ements 
were carried out in Jcffersonville, whereas the purely computerized operations were done at 
the Census Bureau's headquarters in Suitland, Maryland. As needed, data files were 
transmitted electronically between the two locations. 

For two of the operations, clerical review of questionnaires and resolution of rejects from the 
computer pre-edit, Jcffersonville personnel sometimes made callbacks to respondents to try to 
resolve data problems. In particular, for the School Survey this was often done to try to 
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reconcile large differences between reported enrollments and teacher counts and expected 
values of these items based on the CCD for the 1989-90 school year. In Round 1, many 
discrepancies of this kind were detected only after these items were compared at the aggregate 
level (see Section 2.5), and their resolution at that stage caused significant delays in the 
production of clean data files. In Round 2, the CCD values for these items were included on 
the mailing labels for the school questionnaires, so that large discrepancies could be detected 
and resolved by the clerical reviews in Jeffersonville. Any discrepancies not resolved at that 
stage were flagged for resolution by the computer pre-edit. 

The specifications for data entry called for 100 percent verification of all data keyed from the 
questionnaires. 

Checks for invalid entries for specific items, inconsistencies between items and other pr ,blems 
were included in both the pre-edit and edit operations. The difference lies in how these 
problems were resolved. In the pre-edit, a listing of rejected schools and items was produced 
and sent to Jeffersonville, where the clerical staff reviewed the listings in conjunction with the 
questionnaires and, as needed, recontacted respondents. In the edit, problems detected in the 
corrected data files were resolved through programmed instructions to blank or impute 
problem items; there were no attempts to contact respondents at this stage. Some 
questionnaires were rejected in the edit and the schools were treated as nonrespondents if 
values were still missing or out of range for selected key items. 

Imputation At several stages during data processing, respondents' or interviewers' initial 
entries on the questionnaire are changed or deleted, or values (including 0) are supplied for 
items initially left blank on the questionnaire. Except when these changes are the result of 
foilowup contacts with respondents, the process of changing or deleting entries is called 
imputation. Some data items are changed at more than one stage; for example, an item 
failing a consistency check in the computer edit might be blanked at that point and a new 
value supplied in the subsequent computer imputation operation. 

Most of the imputation for both Rounds 1 and 2 of the School Survey was done in a 
computerized imputation operation following the computer edit. Some imputation was done 
during the computer edit and a very limited amount in earlier stages of processing, including 
the initial clerical edit and the clerical resolution of pre-edit rejects (see Exhibit 2.1). Most 
changes during the clerical operations resulted from followup contacts with respondents; 
clerical imputation was permitted only in a few situations where the correct entry was obvious 
from other information on the questionnaire. In Round 2, all School Survey items that were 
missing or failed consistency checks were imputed; in Round 1 there were two items on the 
private school questionnaire that were not imputed: an item on place of operation and one 
concerning staffing patterns (items 7 and 35 on Form SASS-3B). 

Information used for imputation in Round 2 came from several sources: other items on the 
same questionnaire; information from the questionnaire for the LEA in which the school was 
located (public schools only); information for the same school from the sample control file, 
which included expected enrollment and teacher counts from the CCD; and information for 
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other sample schools with similar characteristics. The last of these sources was used in a 
sequential hot deck (donor-based) procedure that matched the school with the missing item to 
the most similar responding school in the same stratum. Similarity was determined on the 
basis of variables such as metropolitan status, percent minority enrollment and size of 
enrollment (NCES, 1992). 

The hot deck method can be illustrated by an example. If a school reported that it had a 
remedial reading program but did not report the number of students served by the program, 
the school's total enrollment would be multiplied by the proportion of enrollment served in 
remedial reading programs at another school with similar characteristics (Kaufman and Huang, 
1993, section 8.2). 

Computer imputation was done in two stages. The first stage consisted of imputations for 
which the missing or inconsistent values could be derived with a reasonable degree of 
assurance from other available data for the same school or school district. The remaining 
items, for which this was not possible, were imputed in the second stage, using the hot-deck 
method. 

All items imputed in either stage of the computer imputation operation were flagged as 
imputed on the final data tapes, including the files that are made available for public or 
restricted use by outside researchers. In Round 2, items imputed in the first stage were 
flagged with code 1, indicating "internal imputation"; those imputed in stage 2 were flagged 
with code 2, indicating "donor based" imputation. In Round 1, only a single code was used to 
distinguish imputed values fro n those based directly on reported data. The imputation flags 
in both rounds reflect only those imputations made during the computer imputation operation. 
Additional information about the imputation procedures for Round 2 is given in Chapter Vlll 
of the Data File User's Manual (Gruber, Rohr and Fondelier, 1993), and detailed item-by-item 
specifications for imputation are available in SASS Specifications Memoranda covering each 
survey and each questionnaire. 

Assignment of locale codes Round 1 reinterviews and cognitive interviews using Round 2 
pretest questionnaires showed that responses to the question, "Which best describes the 
community in which the school is located?" had moderate response variance. The same 
question was used in Round 2, but a separate locale or "urbanicity" code was also developed 
by matching each school's mailing address to Census Bureau geographic files containing 
population density data. Standard Metropolitan Statistical Area codes and urban/rural codes. 
The same locale codes were used for the school and school administrator data files. These 
more rigorously defined locale codes will sometimes differ from the codes based on self- 
reports of community type (Kaufman and Huang, 1993, section 1.4.4; Johnson, 1993). 

Weighting Sample weighting procedures used for the School Survey have three purposes: to 
take account of the selection probabilities at every stage of selection; to minimize biases that 
may result from unit nonresponse; and to make use of available information from external 
sources to improve the precision of sample estimates. 
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The weighting procedures for public, private and Indian schools were quite similar, with 
minor variations at some stages. For each sector, the overall weights were the product of four 
factors: a basic weight; a sampling adjustment factor; a school nonresponse adjustment 
factor; and a frame ratio adjustment factor. 

The basic weight is the inverse of the probability of selection of the school. For 
schools selected with certainty it has a value of 1 .00. 

The sampling adjustment factor is used to take account of special circumstances that 
affected a school's probability of selection. Such circumstances included frame 
duplications not discovered prior to sample selection, including the special case where 
a sample school was discovered to have merged with another one also included in the 
frame. Another instance is the one described earlier, in which some special education 
programs, with operations at several locations, were identified in the frame as single 
schools. In this situation, locations not already included in the frame as regular 
schools were subsampled. 

The school nonresponse adjustment Jactor is applied to the schools for which 
acceptable questionnaires were obtained to compensate for unit nonresponse. The 
factors, which are calculated separately for specified adjustment cells, are the ratios of 
the summed sampling weights (product of the first two factors above) for all eligible 
sample schools in the cell to the summed weights for those that responded. For public 
schools the cells are defined by state, school grade level, enrollment size and 
urbanicity. Cells with small samples and those with high factor values are combined, 
following designated rules. A similar procedure was followed for Indian schools, 
using somewhat different cell definitions and rules for collapsing cells. 

A similar procedure is also used for the private school list and area frames. For the 
list frame, the nonresponse adjustment cells are defined by association group, grade 
level and, for Catholic and "all other" category schools only, by urbanicity. For the 
area frame, they are defined by broad types (Catholic, other religious, nonsectarian and 
unknown), grade level and enrollment size class. Similar rules are used to collapse 
small cells and those with high factor values. 

The frame ratio adjustment factor is used to adjust for differences between expected 
and actual sample sizes. Like the nonresponse adjustment factors, these ratio 
adjustment factors are calculated separately for specified adjustment cells. The factor 
for each cell is the ratio of the total number of schools in the frame in that cell to the 
sample estimate of that number, based on all schools selected, without regard to their 
final response status. Schools that turned out to be ineligible or did not respond had to 
be included in the denominator because no corresponding information was available 
for schools not in the sample. 

For public schools and private schools in the list sample, the cell definitions and 
collapsing rules arc similar, although not always identical, to those used for the 




nonresponse adjustments. For the private school area sample, the frame ratio 
adjustment factors are used only for the PSUs selected with certainty, because there is 
no universe frame for the non-certainty PSUs. For the certainty PSUs, the cells used 
are defined by grade level and PSU. 

Further details on weighting are provided in the Data File User's Manuals and the reports on 
Sample Design and Estimation for each round. 

The nonresponse adjustment procedure is based on the assumption that the probability of 
nonresponse may vary between cells, but does not vary among individual schools within cells. 
Therefore, it is important to define the cells in a way that makes nonresponse probabilities as 
homogeneous as possible within cells. Shcn, Parmer and Tan (1992) explored the correlates 
of nonresponse, based on the Round 2 School Survey samples of public and private list 
sample schools, using a variety of analytical procedures. Their analyses confirmed the 
appropriateness of the adjustment cells currently in use for public schools. They suggested a 
change in the order of variables used in collapsing cells when collapsing is necessary. For the 
private school list frame, they suggested the use of enrollment size in creating adjustment 
cells, in addition to the variables currently used, as well as a change in the order of variables 
used in collapsing cells. Their suggestions have been adopted in part for the Round 3 School 
Survey. 

Variance estimation The balanced half-sample replication method has been used to estimate 
the sampling errors associated with estimates for all of the SASS surveys. Replicates are 
subsamples of the full sample. The statistic of interest, such as number of students at a 
specified grade level, is estimated from each replicate. (In Round 1, the same overall 
nonresponse and frame ratio adjustment factors were used for each replicate. In Round 2 
these factors were calculated separately for each replicate.) The mean square error of the 
replicate estimates around the full sample estimate provides an estimate of the variance of the 
statistic. For the balanced half-sample procedure, each replicate or subsample consists of 
approximately one-half of the full sample of schools. Each sample school is included in one- 
half of the replicates, except for schools selected with certainty, which are included in all 
replicates. 

A total of 48 replicates were designated for each of the SASS surveys. For the School 
Survey, special procedures were used to ensure that the effect of controlling overlap of the 
Round 1 and Round 2 samples would be properly reflected in estimates of variance for 
changes occurring between the two rounds. Details on the procedures for designating the 
replicates are provided in the Data File User's Manuals and the Sample Design and 
Estimation Report for each round. Each SASS public-use data file contains the 48 sets of 
weights needed to produce balanced half-sample replicated variance estimates, so that file 
users can estimate the sampling errors for statistics that are of interest to them. The same 
procedures are used to estimate the sampling errors that are presented in all SASS 
publications. 
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A recent study has confirmed the feasibility of including generalized variance functions in 
SASS publications (Salvucci and Holt, 1992). These functions, which relate the sampling 
error of an estimate to its size, can be used by those who do not w 'ith microdata files, or 
lack the software for the replication method, to produce approxin * ■•? ^ to the sampling errors 
associated with their estimates of interest. Specific parameter values for the four basic 
surveys in Round 1 have been computed (Salvucci, Holt and Moonesinghe, 1994) and will be 
used for internal analyses. Parameter values for Round 2 are being developed and will be 
made available to data users. 

2.5 Evaluation of estimates 

This section describes comparisons of weighted School Survey estimates with data from other 
sources, including other SASS surveys, the CCD and, to a limited extent, data from other 
agencies and organizations. Comparisons of survey estimates with other data can be made 
both prior to and following publication. In Round 1, as soon as weighted data files were 
available, state-level estimates of numbers of schools, total enrollment and teacher counts 
were compared with CCD and QED data. The large differences thai were found for public 
schools in some states led to a substantial amount of review of individual school records and 
questionnaires and, in some instances, recontacts with schools in order to correct erroneous 
data. A similar set of operations was undertaken in Round 2, except that QED data were not 
used in the comparisons. This sequence of pre-publication review and correction operations is 
referred to as the "post-processing edit". 

Round 1 : Post-processing edit An initial comparison of survey estimates and CCD counts of 
total enrollment by state identified several states, predominantly in the Midwest, for which 
public school enrollment estimated from the School Survey was much higher than expected. 
To identify the individual schools that might account for the differences, a list was prepared 
of 972 public schools whose reported enrollment exceeded the expected value, based on the 
QED frame, by 35 percent or more. Questionnaires for 687 of these schools (excluding those 
in the states with the lowest ratios of reported to expected enrollment) were subjected to 
detailed reviews. About one-fourth of the 687 questionnaires were accepted as correct. 
About one-half of them had been partially filled for the school district or for more than one 
school and there was enough information on the questionnaire to make corrections. For the 
remaining one-fourth, it appeared that all questionnaire items had been completed for a school 
district, two schools, or the wrong school. These cases were assigned to NCES and Census 
Bureau staff for telephone reinterviews, which were successfully completed for about four- 
fifths of them. 

This experience led to the following conclusion: 

Our review of the questionnaires and phone conversations with school secretaries and 
principals lead us to believe that these errors were made because the respondents 
misread the first question on the school questionnaire, or because of their employment 
positions (district superintendent, assistant superintendent, principal for two schools, 
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etc.), they assumed we wanted information for all schools under their jurisdiction 
rather than the school named on the questionnaire label. (Fondelier, 1989a) 



After the corrections based on this review had been incorporated in the School Survey data 
file, further comparisons with CCD data showed that estimates of the number of public school 
teachers for some states were still much higher than expected, based on the CCD. As in the 
previous case, a list of suspect schools was compiled, based on several criteria, such as 
student/teacher ratios, comparison of head counts and full-time equivalent (FTE) co mts of 
teachers, and comparison of FTE counts for the school and for the district in which it was 
located. Two of the criteria used for flagging schools were met by more than one-eighth of 
all sample schools. 

In this instance, the review was based primarily on examination of computer listings and a 
review of programming specifications for prior operations. There were no recontacts with 
schools and only a few of the original questionnaires were examined. The review led to 
corrections for 281 school records. The conclusion concerning sources of the problem was: 

The problem of FTE teacher overestimates was caused chiefly by the respondents' 
reporting district enrollment for some schools - the inconsistency between the district 
enrollment and the school teacher count triggered the edit procedure which increased 
the teacher count [which had not been subsequently returned to its original value when 
the enrollment count was corrected]. Other causes were probably poorly recorded 
entries which were misread by the keyers and a lack of understanding by some 
respondents of "full-time equivalent." (Fondelier, 1989b) 

The review found that 13.7 percent of the interviewed schools with one or more part-time 
teachers had identical entries for the head count and FTE number of teachers. However, no 
changes were made for these schools; it was believed that "... changing the FTE entries for 
these cases would not significantly alter the weighted teacher counts for the states in which 
they were located" (Fondelier, 1989b) 

Round 1: Evaluation of published estimates Information obtained from pre-publication 
reviews, comparisons with CCD data, and other sources of information within NCES leads to 
the following conclusions about the quality of final estimates from the School Survey in 
Round 1*. 

School counts estimated from SASS were lower than those obtained from NCES's 
Public Elementary/Secondary School Universe Survey, a part of the CCD program. At 
the rational level, CCD counts exceeded SASS estimates by 6 percent, with the 
di ences being greatest for Nebraska (44 percent), North Dakota (42 percent). South 
Dakota (38 percent) and Montana (15 percent). These differences were due in part to 
definitional differences between the QED (frame for the Round i School Survey), 
which defines schools in terms of physical locations, and the CCD, which defines them 
in terms of administrative units (NCES, 1991a). 
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• In Nebraska, the QED was found to have excluded some small (elementary grades 
only) LEAs with a total of about 275 schools and 2,800 students. The schools, 
students and teachers in these LEAs were not included in SASS (Hammer and Gerald, 
1991, p.22). 

• The PTE teacher counts from the School Survey are likely to be overestimates. In the 
average state, 19 percent of the schools having part-time teachers reported identical 
FTE and head counts for teachers (NCES, 1991a). 

• FTE teacher counts rbr Hawaii from SASS were substantially higher than the CCD 
counts. For the latter, the state had reported in terms of "authorized" positions that 
were filled, whereas in SASS the schools correctly reported all teachers, regardless of 
whether their positions were officially authorized (Fondelier, 1989b). 

A report by an NCES contractor (Smith and Salvucci, 1989) compared preliminary estimates 
of private school enrollment from the Round 1 School Survey with estimates from the 
October Education Supplement to the Census Bureau's Current Population Survey (CPS) and, 
for Catholic Schools, compared SASS estimates of school counts and enrollment with data 
available annually from the National Catholic Educational Association (NCEA). 

The CPS estimate of enrollment in private elementary and secondary schools in October 1987 
was 4,420,000. This estimate was 16.5 percent below the preliminary SASS estimate of 
5,291,000 and 15.3 percent below the final SASS estimate of 5,218,000. CPS estimates of 
private school enrollment were also significantly below estimates from NCES sources other 
than SASS for 1983, 1985 and 1988. The report asserted that "... these differences cannot be 
fully explained without a major benchmarking study" and recommended that if such a study 
were undertaken it should be done when data from the 1990 Census of Population were 
available. 

For Catholic private schools, the SASS estimates of number of schools and enrollment for the 
school year 1987-88 both exceeded the NCEA figures, which are based on an annual census 
covering all schools identifiable as Catholic, including those operated by private boards of 
control and not affiliated with a parish or diocese. 

1987-88 estimate of: NCEA SASS 



Preliminary 



Final 



Number of Catholic 
schools 



8,992 



9,540 



9,527 



Enrollment (thousands) 



2,623 



2,827 



2,823 
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As shown, the SASS final school and enrollment estimates exceeded those of the NCEA by 
5.9 and 7,6 percent, respectively. For the following school year, the estimate of Catholic 
school enrollment from NCES's Early Estimates Survey exceeded the NCEA's count by 9.2 
percent, A subsequent review of school lists available annually from the Council for 
American Private Education (1992) suggested that the higher estimates from SASS may be 
accounted for in part by schools that are affiliated with the United States Catholic Conference, 
but were not included in the NCEA annual census. 

Round 2: Post-processing edit (For additional details, see Chapter VII, Section F of the Data 
File User's Manual for Round 2.) In Round 1 the discovery, at the weighting stage, of 
discrepancies between SASS preliminary estimates and CCD counts of schools and teachers 
led to substantial unanticipated processing costs and significant delays in publication of the 
survey results. Changes were introduced in Round 2 in an attempt to eliminate or minimize 
the impact of such problems. The CCD replaced the QED list as the primary source of the 
frame for the public school sample. Instructions were added to the public school 
questionnaire to report data only for the school named on the label, and the expected number 
of teachers and students for each school ^'as displayed on the label. Expected and reported 
school enrollment and teacher counts were compared in the field office edit operation, with 
followups for differences of 50 percent or more. 

In spite of these changes, initial post-processing comparisons of weighted estimates by state 
showed that the SASS estimates of total teachers from the public school data file for 9 slates 
were at least 15 percent greater than the state FTE teacher counts from the 1991 CCD, and 
staff reviews identified significant data problems in one additional state. For these 10 states, 
approximately 375 schools with large differences between the SASS and CCD records were 
identified. The individual records for these schools were compared and, when appropriate, the 
SASS records were changed to make them consistent with the CCD data for the schools. 
Changes were made to about 300 of the SASS records in this group. These comparisons of 
records for individual schools showed that there were two main causes of the SASS 
overestimates of teachers: schools that reported data for all of the schools in a school district 
and instances where there were 2 or more schools (as defined for CCD) at a single location 
and they had been reported as a single school. 

After the changes were made, the school files for the 10 states were reprocessed to produce 
new estimates of students and teachers. Some residual problems were identified and a few 
additional changes were made. As will be noted in Chapter 5, for the schools for which 
changes had been made in the post processing edit, some of the teacher records also required 
changes. 

Round 2: Evaluation of published estimates Final estimates of public schools by state were 
compaied with school counts from the CCD. The SASS estimate for one state, Oklahoma, 
was about 15 percent higher than the CCD count. There were differences of 5 to 10 percent 
for 8 states and the District of Columbia, The SASS estimates were from 5 to 10 percent 
higher for Arizona and Nevada and 5 to 10 percent lower for Alaska, Louisiana, Minnesota, 
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South Dakota, Wisconsin, Wyoming and the District of Columbia. For the remaining 42 
states, the SASS estimates were within 5 percent of the CCD counts. For the United States, 
the SASS estimate was 97.9 percent of the CCD count for the same school year (Gruber, 
Rohr and Fondelier, 1993, Table XII-4). 

We would expect the CCD counts to be higher mainly for two reasons. With a few 
exceptions, the SASS sample did not cover schools that did not exist at the time of the 1989 
CCD, but were reported in the 1991 CCD. Second, as Table 2.8 shows, 4.0 percent of the 
schools sampled from the 1989 CCD were found in SASS to be ineligible for the survey 
because they were no longer operating, had merged with another school or were not serving 
students in any of grades 1-12. A factor causing differences in the other direction was that 
some of the schools sampled from the 1989 CCD list were found in SASS to represent more 
than one school, for example, an elementary and secondary school operating at the same 
location but under separate administration. The elimination of schools that were ineligible 
during the reference school year and the inclusion in the sample of units with more than one 
school did not cause any bias in the survey estimates. 

The SASS estimates of the number of private schools by school type (9 categories of schools 
with religious affiliations and non-sectarian schools) were compared with counts from the 
1989-90 Private School Survey, which provided the sampling frame for the sample of private 
schools for the Round 2 School Survey (which covered the 1990-91 school year). The SASS 
estimates were smaller in all 9 categories: this was primarily the result of PSS schools that 
were found to be out of scope in SASS. As Table 2.8 shows, 5.6 percent of the sample of 
private schools were found to be ineligible, because they were no longer operating, had fewer 
than 10 students, or did not meet the SASS definition for other reasons. For the United 
States, the SASS estimate of the number of private schools came to 92.4 percent of the 1989- 
90 PSS count (Gruber, Rohr and Fondelier, 1993, Table XII-5). 

For private schools, the Round 2 School Survey estimates of numbers of students and teachers 
were compared with counts from the 1989-90 PSS. The SASS student counts were 3.8 
percent higher than the PSS counts and the SASS teacher counts were 1.7 percent lower. 
These differences may have been due in part to the sampling error associated with the SASS 
estimates. 

The 1989-90 PSS enrollment counts for Catholic schools exceeded counts from the National 
Catholic Education Association (NCEA)'s census for the same year by 5.4 percent of the 
latter's figure (Gruber, 1992b). Differences by state showed large variations; however, some 
of these could be attributed to the inclusion by the NCEA in a single state of counts for 
archdioceses with schools in more than one state. 

Data reported on the School Survey questionnaires were not always internally consistent. For 
example, the total of enrollment counts by grade frequently differed from the total of 
enrollment counts by r?ce for the same school. The questionnaire item on enrollment by 
grade asked for counts as of October 1 , whereas the item on enrollment by race did not 
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specify a reference date. For schools with large discrepancies, the data for the two items were 
edited to make them consistent, but there were some residual differences. 
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Table 2.1 School Survey Mail Response as a Percent of Total Response (List Frame 
Only): Round 2 



Type of School 

Uj ul dUt LtVLI 


Percent Obtained by Mail* 
(Percent) 


Public 


Private 


Elementary 


67.5 


60.3 


Secondary 


67.2 


57.7 


Combined (Elementary & Secondary) 


66.8 


47.7 


TOTAL 


67.3 


55.7 



♦Mail responses as a percent of mail plus telephone responses. 
Source: Parmer, Shen, and Tan (1992). 



Table 2.2 School Survey Mail Response as a Percent of Total Response by Metropolitan 
Status (List Frame Only): Round 2 



Metropolitan Status 


Percent Obtained by Mail* 
(Percent) 


Public 


Private 


Large Central City 


54.9 


54.6 


Mid-size Central City 


66.4 


Urban Fringe of Large Central City 


65.2 


54.2 


Urban Fringe of Mid-size Central City 


69.5 


Large Town - Non MSA** 


73.7 


63.1 


Small Town 


71.4 


Rural 


67.0 


TOTAL 


67.3 


55.7 



♦Mail responses as a percent of mail plus telephone responses. 
♦♦Metropolitan Statistical Area. 
Source: Parmer, Shen, and Tan (1992). 
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Table 2.3 School Survey Mail Response as a Percent of Total Response for Public 
Schools: Round 2 (Shown in Percent) 



State 


Percent 
Obtained 
by Mail* 


1 

State 


Percent 
Obtained 
by Mail* 


A 1 'J l\Q TT1Q 

rMdUdlild 


71. 1 


Montana 


64.7 


A loclfa 


60.0 


Nebraska 


69.5 


\ pn r\ n n 

a rizuiid 


59.9 


Nevada 


71.6 


Arkansas 


68.7 


New Hampshire 


59.3 




61.3 


New Jersey 


55.5 




57.9 


New Mexico 


60.1 


^onne ucui 


69.2 


New York 


62.5 


uemwd re 


81.1 


North Carolina 


69.8 


LllbinCl 01 \_OlUlTJUId 


47.9 


North Dakota 


67.3 


r lOrlQH 


74,8 


Ohio 


64.8 


Lieorgid 


68.9 


Oklahoma 


59.2 


lid Wall 


70.7 


Oregon 


70.3 


Frt ihn 
luallU 


75.2 


Pennsylvania 


68.8 


Fill nn 5 c 

HllllUlS 


67.0 


Rhode Island 


60.7 


Inula 113 


77.6 


South Carolina 


71.0 


Iowa 


65.8 


South Dakota 


63.6 


Kansas 


68.5 


Tennessee 


70.6 


Kentucky 


72.5 


Texas 


64.4 


Louisiana 


67.0 


Utah 


77.0 


\yf aina 
lYldillC 


71,0 


Vermont 


75.2 


ivi dry tdiiu 


72.5 


Virginia 


79.3 


Massachusetts 


69.1 


Washington 


69.3 


Michigan 


59.2 


West Virginia 


77.9 


Minnesota 


61*2 


Wisconsin 


74.1 


Mississippi 


67.4 


Wyoming 


72.2 


Missouri 


67.8 


TOTAL 


67.3 



♦Mail responses as a percent of mail plus telephone responses. 
Source: Parmer, Shen, and Tan (1992). 
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Table 2.4 School Survey Mlail Response as a Percent of Total Response for 
Private Schools {List Frame Only): Round 2 



Association Group 


Percent 
Obtained 
by Mail* 


Total 


55.3 


Association of Military Colleges and Schools - US 


66.7 


Catholic 


63.0 


Friends 


42.3 


Episcopal 


50.5 


National Society for Hebrew Day Schools 


35.1 


Solomon Schecter 


42.5 


Other Jewish 


36.1 


Lutheran - Missouri Synod 


73.6 


Evangelical Lutheran Church - Wisconsin Synod 


66.0 


Evangelical Lutheran Church in America 


7L3 


licit* 1 irfhofon 


5X 2 


CiCYcll lll-Udj /VUYLIIWMo 


57 0 


l>ll(!M!dll JiLIJUUib tlllcl lldliUIIdl 


64 0 


American Association of Christian Schools 


30.7 


National Association of Private Schools for Exceptional 
Children 


58.1 


Montessori 


48.5 


National Association of Independent Schools 


48.8 


All Other 


50.3 



♦Mail responses as a percent of mail plus telephone responses. 
Source: Parmer, Shen and Tan (1992). 
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Table 2.5 School Survey Response Rates 





Round 1 (1988) 


Round 2 (1991) 


Unweighted 


Weighted 


Unweighted 


Weighted 


Public 


91.9 


91.9 


95.0 


95.3 


Private 


79.6 


78.6 


85.1 


83.9 



Sources: 

Round I Unweighted: Kindel (1989). 

Round 1 Weighted: NCES (1991c). 

Round 2: (iruber, Rohr and FontKier (1993). 
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Table 2.6 School Survey Weighted Response Rates for Public Schools by State 





Response Rate 




Response Rate 


State 


Round 1 


Round 2 


State 


Round 1 


Round 2 


Alabama 


96.6 


95.9 


Montana 


94.8 


97.8 


Alaska 


96.9 


92.0 


Nebraska 


96.4 


98.7 


Arizona 


97.0 


94.8 


Nevada 


96.1 


96.1 


Arkansas 


95.1 


97.7 


New Hampshire 


97.0 


96.3 


California 


88.0 


94.6 


New Jersey 


91.5 


88.3 


Colorado 


99.1 


95.9 


New Mexico 


883 


96.0 


Connecticut 


88.6 


93.1 


New York 


84.9 


87.6 


Delaware 


91.0 


933 


North Carolina 


90.5 


92.6 


District of Columbia 


68.0 


86.3 


North Dakota 


100.0 


98.4 


Florida 


97.5 


93.9 


Ohio 


95.0 


97.0 


Georgia 


95.0 


96.6 


Oklahoma 


89.5 


96.3 


Hawaii 


77.7 


98.7 


Oregon 


96.6 


95.3 


Idaho 


98.2 


98.6 


Pennsylvania 


87,0 


96.1 


Illinois 


95.4 


98.7 


Rhode Island 


99,1 


96.5 


Indiana 


97.2 


99.6 


South Carolina 


88.3 


96.6 


Iowa 


95.9 


96.5 


South Dakota 


94.8 


98,5 


Kansas 


93,1 


98.0 


Tennessee 


91,9 


98.1 


Kentucky 


90.0 


98.1 


Texas 


87,1 


97.4 


Louisiana 


88.8 


93,9 


Utah 


100.0 


98.4 


Maine 


97.1 


94.7 


Vermont 


99.3 


98.5 


Maryland 


74.5 


81.0 


Virginia 


89,9 


92.2 


Massachusetts 


94.6 


91.1 


Washington 


99.5 


92,6 


Michigan 


97.5 


97.1 


West Virginia 


94,4 


98.2 


Minnesota 


91.4 


97,4 


Wisconsin 


93.5 


94.6 


Mississippi 


96.7 


97,2 


Wyoming 


93,5 


97.7 


Missouri 


85.4 


98,0 


TOTAL 


91.9 


95.3 



Sources: NCKS (1991c) and Oruber, Rohr and Kondelier (1993). 




Table 2.7a School Survey Weighted Response Rates for Private Schools 
by Association Group: Round 1 



j Response Rate 
Association Group | (Percent) 


Total 


78.6 


Area Sample 


66.9 


Association of Military Colleges and Schools - US 


86.4 


Catholic 


89.8 


Friends 


83.0 


Koiscooal 


82.0 


Jewish 


71.9 


Lutheran 


90.3 


Seventh-day Adventists 


88.7 


Christian Schools International 


95.3 


American Association of Christian Schools 


55.6 


National Association of Private Schools for Exceptional 
Children 


83.5 


American Montessori Society 


82.0 


National Association of Independent Schools 


73.6 


Other 


70.6 



Source: NCES (199?c). 
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Table 2.7b School Survey Weighted Response Rates for Private Schools 



by Association (Jroup: Round 2 


1 


Association Group 


Response Rate 
trcrceni) 


Total, area frame and list frame 


83.9 


Area frame 


74.0 


Association list frame 


wm ■ 


Association of Military Colleges and Schools 


90.9 


National Catholic Education Association, Jesuit Secondary 
Education Association 


90.9 


Friends Council on Education 


90.6 


National Association of Episcopal Schools 


89.4 


Hebrew Day Schools 


mm r\ r i 

70.8 


Solomon Schechter Day Schools 


85.1 


Other Jewish 


70.4 


Lutheran Church- Missouri Synod 


96.1 


Evangelical Lutheran Church-Wisconsin Synod 


97.9 


Evangelical Lutheran Church in America 


95.5 


Other Lutheran 


94.2 


General Council of Seventh-day Adventists 


93.9 


fMtricfinn Qr»li/wklc Int Prnsif if inn! 


93.7 


A marinnn A ccf i^tof inn /if f r*icf i Q n ^f»h*Wilc 

International 


59.0 


National Association of Private Schools for Exceptional 


86.5 


American Montessori Society Schools 


85.5 


National Association of Independent Schools 


84.6 


All else 


81.1 



Source: (iruber, Ruhr and Fondelicr (1993). 
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L-fold Index 
of Inconsistency 


Round 2 
(1991) 


37.6* 
34.7-40.9 








Round 1 
(1988) 


42.4* 
39.6-45.4 








Index of 
Inconsistency 


Round 2 
(1991) 




45.1 

39.0-52.3 


30.1* 


24.7 

20.5-29.7 


Round 1 
(1988) 




53.5 
47.7-60.0 


i-H 


31.7 

26.8-37.4 


Gross Difference 
Rate 


Round 2 
(1991) 


30.4* 

27.9-32.9 


12.1* 
10.5-14.1 


13.7* 
12.0-15.8 


ND 

oo o 

QO 3 


Round 1 
(1988) 




16.2* 
14.5-18.2 


16.1* 


9.3 
7.9-11.0 


Percent Yes 
(Survey Interview) 


Round 2 
(1991) 1 




14.2 


28.3 


23.0 


Round 1 
(1988) 




15.3 


31.6 


16.3 


Topic 

(Text of questions is 
presented below) 


Community Where 
School Is Located 
Point Estimate 
90% Confidence 
Interval 


Bilingual Education 
Point Estimate 
90% Confidence 
Interval 


English as a Second 
Language 

P' .it Estimate 
| y0% Confidence 
Interval 


Extended Day Care 
Point Estimate 
90% Confidence 
Interval 
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Exhibit 2.1 



Major Processing Steps 
for SASS Surveys 



Activity 


Location* 


Jeffersonville 


Suitiaad 


Computer check-in of questionnaires 


X 




Clerical review of questionnaires** 


X 




Data entry 


X 




Merge data and sample control files 




X 


Computer pre-edit 




v 
A 


Resolution of ore-edit reiects** 


X 




Input of corrections to data file 




X 


Computer edit 




X 


Imputation 




X 


Weighting 




X 


Post processing edit 




X 



* Data files are transmitted electronically between the Census Bureau's processing facility in 
Jeffersonville, Indiana and headquarters in Suitland, Maryland. 

** Includes telephone follow-ups to respondents as needed. 
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CHAPTER 3 



THE SCHOOL ADMINISTRATOR SURVEY 



3*1 Introduction 

With a few exceptions, there is a one-to-one correspondence between the SASS samples of 
schools and school administrators. The goal of the School Administrator Survey is to collect 
information from the principal, headmaster or headmistress of each school selected for the 
School Survey. School administrators are asked to complete their own questionnaires, 
whereas for the school questionnaires it is acceptable for a principal to delegate all or part of 
the job to a staff member. 

This chapter is organized in the same way as the preceding chapter on the School Survey. It 
has four sections covering the main phases of the survey operations: frame development and 
samplmg (3.2); data collection procedures and associated errors (3.3); data processing and 
estimation (3.4); and evaluation of estimates (3.5). However, the chapter will be much shorter 
than Chapter 2, because many details of the relevant design features and procedures have 
already been described in Chapter 2 and will not be repeated in this chapter. 

In Round 1 of SASS a single questionnaire, SASS-2, was used for both public and private 
school administrators; however, there were minor differences in the public-use data files 
produced for the two sectors. For example, the class interval codes assigned to 
administrators' salaries represented different ranges for the public and private schools. In 
Round 2, separate questionnaires, the SASS-2A and SASS-2B, were used for the public and 
private sectors. Most of the data items on the two versions were the same. 

The content of the School Administrator questionnaires was similar for Rounds 1 and 2, The 
majority of the items are about the administrators' demographic characteristics, training, 
experience, salary and benefits. The remaining items request the administrators' views on 
such topics as: the relative seriousness of different kinds of problems affecting the school; the 
relative influence of the administrator and others, including teachers, parents and school 
boards, on their schools' policies and activities; and, in Round 2 only, the quality of the 
teaching staff and the relative importance of different educational goals. 
A few changes in content occurred between Rounds 1 and 2, Round 1 questions that were 
dropped covered a breakdown of time spent by the administrator on different kinds of school- 
related activities, programs for teacher evaluation and assistance to beginning teachers, and 
problems encountered in filling vacancies. New questions in Round 2 dealt with educational 
goals and the quality of the teaching staff. In addition, several new items were added to the 
list of school problems whose relative seriousness the administrators were asked to evaluate. 

3.2 Frame development and sampling 

The target population for the School Administrator Survey consists of the principals or head 
administrators of all public and private schools eligible for inclusion in the School Survey. A 
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few of these schools do not have administrators. Recipients of the School Administrator 
Survey questionnaire for such schools are asked to check a box for "School has no 
administrator" and return the questionnaire to the Census Bureau. In Round 1, according to 
published survey estimates (Choy, Medrich, Henke and Bobbin, 1992, Tables 2,1 and 3.1), 
1 percent of the public schools and 5 percent of the private schools had no administrator. 
Schools with low enrollment and those in rural areas were less likely to have an administrator. 

Once the sample of schools is selected, no additional sampling is needed to select the sample 
of school administrators, A detailed description of the frame development and sampling 
procedures for schools is provided in Chapter 2 ? Section 2,2. 

3.3 Data collection procedures and associated errors 

Data collection procedures for Round 2 As described for the School Survey in Chapter 2, 
Section 2.3, the data collection procedure for SASS began with advance mailings to LEAs 
and school principals to explain the nature of the SASS data collection activities and, in the 
case of the principals, to ask them to submit a list of teachers for use in selecting the sample 
of teachers for the Teacher Survey. The questionnaires for the School and School 
Administrator Surveys were mailed to the principals in December 1990. Nonrespondents 
were followed up by mail in January 1991 and those who did not respond to the second 
mailing were followed up by telephone during the second quarter of 1991. Because of the 
content of the School Administrator Questionnaire, it was important that the questionnaire be 
completed by the administrators themselves. An instruction at the beginning of the 
questionnaire reads: 

It is important that this questionnaire be completed by the school administrator (i.e., 
the principal or head), not by anyone else at the school. 

Supervision and quality assurance As in the case of the School Survey, the two primary 
methods of controlling the quality of the data collection operations were regional office 
reviews of the questionnaires completed by Census Bureau field representatives in their 
telephone followups of nonrespondents and reinterviews of both mail and telephone 
respondents for a sample of completed questionnaires. The procedures for the regional office 
reviews were identical to those used for the School Survey, as described in Chapter 2, Section 
2.3. 

For both rounds of SASS, reinterviews were attempted for about 10 percent of the school 
administrators. They were successfully completed for 87 percent of eligible cases in Round 1 
and for 94 percent in Round 2. All reinterviews of school administrators in both rounds were 
conducted by telephone. For Round 1, a single reinterview questionnaire was used to re-ask 
selected items from both the school and administrator questionnaires. For Round 2, separate 
reinterviews of school administrators were conducted covering a subset of items from the 
School Administrator Questionnaire only. Results are presented below, under the heading 



"Measurement error, findings from re interviews". 



Nonresponse error As explained in Chapter 2, a few LEAs requested NCES not to ask 
sample schools in their district to participate in SASS, so no questionnaires were obtained for 
the administrators of these schools. In Round I, the nonresponse from this source was less 
than 0.5 percent of the eligible school administrators (Nash, 1988). 

Most of the unit nonresponse was associated with individual school administrators. Tables 3.1 
to 3.3 show unweighted and weighted response rates for Rounds 1 and 2, for public schools 
by state and private schools by association group. Administrators of schools not operating in 
the school year of reference for the survey or that failed to meet the definition for other 
reasons are excluded from the base of the response rates. Table 3.4 shows, for public and 
private schools in Round 2, the percentages of the initial sample that were excluded for such 
reasons. 

As shown in Table 3.1, weighted response rates for public school administrators were higher 
than those for private school administrators in both rounds, however, the gap narrowed 
substantially between Rounds 1 and 2. For public school administrators the weighted 
response rate in Round 2 was 2.3 percentage points above the corresponding figure for Round 
1. For private school administrators, the increase between Rounds 1 and 2 was 10.8 
percentage points. 

Within each sector there was substantial variation. In Round 1, in the public sector, 29 states 
had weighted response rates of 95 percent or better and only 1 (the District of Columbia) was 
below 80 percent (Table 3.2). In Round 2 there were 42 states with weighted response rates 
of 95 percent or better and the lowest response rate was 82.4 percent, for Maryland. In the 
private sector, the range of weighted response rates by association group in Round 1 was from 
56.1 percent to 97.9 percent. Most groups were in the range from 70 to 90 percent (Table 
3.3a). In Round 2, with an expanded set of association groups, the range of weighted 
response rates was from 72.4 to 98.9 percent and 14 of the 18 groups had rates above 90 
percent (Table 3.3b). 

Data on unweighted item response rates for the School Administrator Survey in Rounds 1 and 
2 are shown in Table 3.5. In general, item response rates were high for both public and 
private school administrators in both rounds. Because of changes in content, data for the two 
rounds are not directly comparable, but they suggest that response to individual items was 
somewhat better in Round 2. 

The lowest response rates in Round 1 (70.3 percent for public schools and 72.3 percent for 
private schools) were for a multiple-response item (mark all that apply) asking about methods 
of compensating for unfilled teacher vacancies. This item was not included in Round 2. The 
items reported as having low response rates in Round 2 were items immediately following 
skip instructions, i.e., they did not apply to all respondents. A review by Jenkins (1992a) 
indicated that these same items were answered by some respondents to whom they did not 
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apply, and suggests that some respondents may have misinterpreted the skip instructions. 

Measurement error: findings from reinterviews {Note: For the following discussion of 
reinterview results, readers not familiar with the interpretation of statistical measures of 
response variance developed from reinterviews may wish to refer to the side bar explaining 
these measures, in Chapter 2, p. 2.14.) As stated earlier, reinterviews were conducted, 
covering selected items from the questionnaire, for about 10 percent of the school 
administrators in both Rounds 1 and 2. Only one topic, the administrators' college degrees 
and major fields of study, was included in the reinterviews for both rounds. The Round 1 
reinterviews had shown unexpectedly high response variances for reports of bachelor's and 
master's degrees by school administrators. As a consequence, the format of the questions for 
this topic was substantially revised. In Round 1, a multiple-response (mark all that apply) 
format had been used to cover all types of degrees. In Round 2, separate sets of questions 
were asked about bachelor's and master's degrees. 

Table 3.6 shows the reinterview measures of response variance for reports of bachelor's and 
master's degrees for both rounds, for public and private school administrators combined. 
Gross difference rates for both items were substantially lower in Round 2, as was the index of 
inconsistency for reports of master's degrees. There were so few principals who did not 
report bachelor's degrees in Round 2 that a reliable estimate of the index of inconsistency 
could not be obtained. In Round 2, the questions about receipt of degree and year of receipt 
all had low response variability (Royce, 1992, Table B), but items on major and minor fields 
of study exhibited response variability in the moderate to high ranges (indexes of 
inconsistency in the range 20 and over). 

Table 3.7 shows the distribution of indexes of inconsistency for all items included in the 
School Administrator Survey reinterviews in Rounds 1 and 2. Response variability for most 
of the items included in the reinterviews has been relatively high, with only a few factual 
items in the low range (under 20). None of the 22 opinion items evaluated in Round 1 had 
low indexes of inconsistency and most were in the high range (over 50). These 22 items 
were of two kinds: 

A set of 13 items asking principals for their views of the relative importance in their 
schools, on a 4-point scale, of each of 13 different kinds of problems that occur in 
some schools. Three of these problem types - student pregnancy, student use of 
alcohol, and student drug abuse - had estimated indexes in the moderate (20 to 50) 
range; the rest were in the high range. 

• A set of 9 items asking principals for their evaluation, on a 6-point scale, of the 
relative influence of teachers, principals, and governing bodies on policies for 
establishing curriculum, hiring new teachers, and discipline. All of these had indexes 
in the high range. 

These two sets of items were retained in the Round 2 School Administrator Survey (both in 
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expanded form), but they were not included in the Round 2 reinter views, based on a belief 
that reinterview results for factual items would provide more information of value for question 
improvement through cognitive research and better questionnaire design (Bushery, Royce and 
Kasprzyk, 1992). 

In general, the reinterview results do not show any significant differences in measures of 
response variability for public and private school administrators. 

Measurement error: findings from other sources In chapter 2, we described findings from in- 
depth interviews, using cognitive research techniques, with school administrators who 
completed the School Survey questionnaire. However, this technique has not yet been used to 
evaluate the School Administrator Survey questionnaires. 

Some changes were made in the Round 2 School Administrator questionnaires based at least 
in part on findings from a review of 600 questionnaires from a pretest conducted in school 
year 1989-90 (Jefferson-Copeland and Bynum, 1990). The format and placement of codes for 
major and minor fields of study at the bachelor's and master's levels were revised. A 
question on hours spent on school-related activities during the most recent full week was 
dropped. Skip patterns for a question on retirement plans were introduced because it was 
determined that the second part of the question was not applicable to all respondents. 

During the data collection for Round 2, regional office staff reviewed a sample of the 
questionnaires completed by telephone followup (Pasqualucci, 1991). An analysis of the 
forms used to record the results of these reviews showed that the main source of errors 
identified was that several of the Census field representatives had failed to record codes for 
major and minor degree fields. Also, a separate item (Check item A) designed to skip the 
next item on the questionnaire when it did not apply had been left blank on several 
questionnaires. 

Further evidence of problems with skip patterns was provided by a review of pre-edit reject 
rates, edit change tallies and post-edit item response rates (Jenkins, 1992a). These data 
showed that the check item referred to in the preceding paragraph had not been completed by 
about 10 percent of the respondents or follow-up interviewers. On the other hand, the edit 
change tallies showed that respondents had apparently failed to follow all of the skip 
instructions and consequently answered some items that did not apply to them. 

3.4 Data processing and estimation 

Data processing procedures The sequence and nature of the data processing operations for the 
Round 2 School Administrator Survey questionnaires were essentially the same as described 
for the School Survey in Chapter 2, Section 2.4. There was one significant difference in 
Round 1: most items missing on the School Administrator Survey questionnaire were not 
imputed, whereas most missing items on the School Survey were imputed. Missing items for 
all of the 4 basic surveys were imputed for Round 2. 
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Imputation in Round 2 Computer imputation of items missing from the school administrator 
questionnaires took place after completion of imputation of items missing from the school 
questionnaires. The purpose of this sequence was to achieve consistency between the school 
and school administrator data for each school. Certain items were common to both 
questionnaires, and the first step in computer imputation of missing items for school 
administrators was to carry over values for these common items, whether reported or imputed, 
from the school records. 

Following this first step, computer imputation for the remaining missing or inconsistent items 
for school administrators proceeded in two stages: logical imputation based on other items 
reported for the same school administrator, following defined rules, and hot deck imputation 
based on responses for other school administrators with similar characteristics. Specific 
details are provided in Chapter VIII of the Data File User's Manual and in SASS 
Specifications Memoranda. 

Imputation in Round 1 For Round 1 of the School Administrator Survey, there was no 
computer imputation following the computer edit- Some items were imputed as part of the 
computer edit and in preceding operations, but all of these imputations were based on other 
information available for the same administrator or schooL There was no hot deck (donor- 
based) imputation of missing values. No imputation flags were included on the final data 
tapes. 

Weighting Weighting procedures for the School Administrator Survey records were the same 
as those used for the School Survey records, using overall weights that were the product of 
four factors: a basic sampling weight; a sampling adjustment factor; a school administrator 
nonresponse adjustment factor; and a frame ratio adjustment factor. Details are provided in 
Chapter 2, Section 2.4. 

School and school administrator weights were developed independently, for two reasons: 
there were some schools that had no administrators and there were some instances in which a 
questionnaire was obtained for the school administrator but not the school, or vice versa. 
These differences meant that nonresponse adjustment factors would not always be the same 
for schools and school administrators in the same cell (NCES, 1991a; Gruber, Rohr and 
Fondelier, 1993). 

Variance estimation A balanced half-sample replication variance procedure (see Chapter 2, 
Section 2.4 for details) is used to estimate sampling errors for all SASS surveys. Replicate 
weights for use in such estimates of sampling error are included on all SASS public-use 
microdata files. As noted above under weighting, there were some schools for which a school 
questionnaire but no administrator questionnaire was obtained, and vice versa. For this 
reason, in Round 1 the replicates and replicate weights were developed independently for 
schools and school administrators, using the same general rules (Kaufman, 1991). However, 
in Round 2 the replicates for school administrators are the same as those used for their 
schools (Kaufman and Huang, 1993). 
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3.5 Evaluation of estimates 



For Round I, SASS estimates of public and private school administrators were compared, 
prior to publication, with administrator and school counts from other sources, including the 
1985-86 Private School Survey, the 1987-88 Common Core of Data, the Quality Education 
Data File (which served as the frame for the Round 1 School Survey), and a figure on the 
number of public school principals from a list compiled by a commercial market data firm 
(Hammer, 1989b). Differences among the estimates were relatively small and were deemed to 
be accounted for by differences in definition and time reference among the estimates 
examined. Differences between SASS estimates of number of schools and number cf 
administrators in both sectors were accounted for primarily by the existence of schools with 
no administrators. 

To check on the quality of SASS estimates of public school principals' salaries, state 
education agencies in several states were asked to provide independent information on average 
principals' salaries in their states (Hammer, 1989a). Four states, Alabama, Illinois, Kansas 
and Maryland, provided information, but in general it was either not directly comparable with 
SASS estimates or there was not enough supporting documentation to determine the extent of 
comparability, Kansas, for example, provided figures 
that included the value of fringe benefits. 

A pre-publication review of estimates based on a set of items about hours spent by 
administrators on school related activities led to a recommendation, which was followed, that 
these estimates not be included in publications (Hammer, 1990). Three factors were cited as 
possibly leading to under-reporting of hours: there was no imputation for individual items for 
which there was no response; there was no "other" category in which to report hours no* 
covered by the named activity categories; and the set of items did not ask for any distinction 
between time spent during school hours and time spent after school hours. 

The Round 2 tabulations for school administrators were inspected and compared with 
corresponding data from Round 1. No unusual differences were noted (Hammer, 1992). 
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Table 3.1 School Administrator Survey Response Rates 





Round 1 (1988) 


Round 2 (1991) 


Unweighted 


Weighted 


Unweighted 


Weighted 


Public 


94.2 


94.4 


96.9 


96.7 


Private 


81.2 


79.3 


91.1 


90.1 



Sources: 

Round 1 Unweighted: Kindel (1989). 

Round 1 Weighted: (NCES 1991c). 

Round 2: Gruber, Rohr and Fondeiier (1993). 
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Table 3.2 School Administrator Survey Weighted Response Rates for Public Schools by 
State 





Response Rate 




Response Rate 


State 


rvounu i 


fxOUFiU it 


State 


Round 1 

IMJUIlU 1 


Round 2 


Ala hum^ 


98.3 


98.9 


Montana 


98.1 


99.8 


A SavLu 


99.0 


96.6 


Nebraska 


95.8 


98.2 


t\ri/Airid 


99.2 


97.1 


Nevada 


96.8 


97.8 




97.0 


96.6 


New Hamnshire 

1 i V 11 a J. U 1 mm villi V 


98.6 


98.8 




92.0 


95.7 


Now ler*;pv 


95.2 


92.4 


f ^rt In r*'J fl * i 


99.2 


98.4 


New Mevii'n 


96.9 


99.2 


rwtrtPfttotti 
v.. Uli 11 v- v 1 lull I 


92.0 


97.0 


New York 


89.1 


89.5 


Tip In w :i rp 


89.7 


94.4 


North Carolina 

1 ^ \M 1 111 CA 1 V 1 1 1 1 CI 


94.2 


95.6 


ISIMllLI Uf v- III Hill U lo 


68.8 


88.9 


North Dakot'i 


95.2 


99.1 


Klnrtdii 

I 11 J 1 IU<1 


99.1 


94.4 


Ohio 


97.1 


97.0 


\ICUI 1" 


95.4 


94.8 


Oklahoma 


90.3 


99.1 


H stwsif i 

1 In >t c* 1 1 


84.8 


98.7 


Oregon 


9/.7 


n*f •-» 

97.3 


Trf tilm 
iUalllJ 


97.2 


100.0 


Pennsylvania 


n f r 

91.5 


97.2 


Illinois 


97.2 


99.8 


Rliode Island 


98.8 


v7.1 


T n d isiii si 

All Ulu llu 


98.4 


100.0 


South Carolina 


90.9 


98.6 


Iowa 


9S.8 


99.0 


Sooth Dakuta 


f nn n 

1UU.0 


98.6 


K linens 


93.9 


98.0 


Tennessee 


94.9 


97.5 


M. Pntt i o\c v 
rv vii i uv, ft. y 


91.7 


99.0 


Texas 


92.4 


no i 

98.1 


1 rttitCftin*! 

1>UU|jI(1 11(1 


91.9 


93.7 


Utah 


t nn n 


nn 4 

99.4 


Maine 


98.7 


98.2 


Vermont 


97.5 


98.6 


1V1 ;-irvln nd 


81.1 


82.4 


Virginia 


93.7 


95.3 


Massachusetts 


92.9 


96.5 


Washington 


98.5 


93.7 


Michigan 


99.1 


98.8 


West Virginia 


95.7 


99.6 


Minnesota 


94.6 


98.8 


Wisconsin 


94.4 


97.2 


Mississippi 


97.6 


97.6 


Wyoming 


88.8 


96.1 


Missouri 


89.5 


98.9 


TOTAL 


94.4 


96.7 



Sources: NCKS (1991c) and Cruber, Rolir and Fondelier (1993). 
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Table 3.3a School Administrator Survey Weighted Response Rates for Private Schools 
by Association Group: Round 1 



Association Group 


Response Rate 
(Percent) 


Total 


79.3 


Area Sample 


66.4 


Association of Military Colleges and Schools - US 


91.7 


Catholic 


90.6 


Friends 


84.9 


Episcopal 


88.1 


Jewish 


71.8 


LfUineran 


88 4 


oevenm-aay Aaveniisis 


OO.O 


v^nrisuan ocnoois international 


97 9 


American Association of Christian Schools 


56.1 


National Association of Private Schools for Exceptional 
Children 


84.6 


American Montessori Society 


78.7 


National Association of Independent Schools 


76.4 


Other 


72.6 



Source: NCES (1991c). 
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Table 3.3b School Administrator Survey Weighted Response Rates for Private 
Schools by Association Group: Round 2 



Association Group 


Response Rate 
(Percent) 


Total, area frame and list frame 


90.1 


Area frame 


83.4 


Association list frame 




Association of Military Colleges and Schools 


95.5 


National Catholic Education Association, Jesuit Secondary 
Fducation Association 


96.2 


Friends Council on Education 


93.8 


National Association of Episcopal Schools 


93.7 


Ht»hrcw Dav Schools 


86.1 


Solomon Schcchter Day Schools 


97.9 


Other Jewish 


72.4 


Lutheran Church— Missouri Synod 


97.3 


Evangelical Lutheran Church— Wisconsin Synod 


97.5 


Evangelical Lutheran Church in America 


98.9 


Other Lutheran 


97.3 


Genera] Council of Seventh-day Adventists 


94,9 


Christian Schools International 


94.3 


American Association of Christian Schools 
International 


73.4 


National Association of Private Schools for Exceptional Children 


94.7 


American Montessori Society Schools 


92.2 


National Association of Independent Schools 


93.7 


All else 


85.0 



Source: G ruber, Rohr and Fondelier (1993). 
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Table 3.5 School Administrator Survey Unweighted Item Response Rates 



Sector 


Ranw of Ifpm 

ixaiigv ui lie 111 

Response Rates 
(Percent) 


Percent of Items with 
Response Rates: 




> 90% 


< 75% 






• 


■ 

Il|§l : 


Public 


70 - 100 


86 


2 


Private 


72 - 100 


89 


2 






Public 


90 - 100 


100 


0 


Private 


80 - 100 


98 


0 



Sources: 

NCES (1991c), 

(Jruber, Rohr and Fondelier (1993). 
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Table 3.7 School Administrator Survey Indexes of Inconsistency- Estimated from 
Reinterviews 



Round and 
Tvt*e of Item 


Number of 
Items 


Index of Inconsistency 


High 
>50 


Medium 
20-50 


Low 
<20 


NA- 
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. .• 
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------- - ^ . .. . • 


Factual 


11 


4 


4 


1 


2 


Opinion 




19 


3 










-:• .; .... . . . 
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-• ' ■ ; '., ■ ' ' ■ 


Factual 


26 


10 


10 


5 


1 



Notes: 



1. Each item either had closed multiple-response categories or was converted to the equivalent by 
assigning class intervals to open-end responses. For items with more than 2 response categories, 
the L-fold index of inconsistency was estimated. 

2. Did not meet the minimum requirements to compute a reliable estimate of the index of 
inconsistency. 

3. No opinion items were included in the Round 2 School Administrator Survey Reinterviews. 
Sources: Newbrough (1989), Royce (1992). 
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CHAPTER 4 

THE TEACHER DEMAND AND SHORTAGE SURVEY 



4.1 Introduction 

The purposes of the Teacher Demand and Shortage Survey are to assess the extent of demand 
for and shortages of elementary and secondary school teachers, identify teacher categories and 
types of schools and school districts for which shortages exist, and collect information that 
can be used to analyze relationships between teacher shortages and the policies of schools and 
school districts with respect to pay, benefits, recruitment and hiring. For the public school 
sector, the information is collected from a sample of school districts; for the private school 
sector it is obtained from a sample of schools. 

This chapter has the same organization as the two preceding chapters. It has four sections 
covering the main phases of the survey operations: frame development and sampling (4.2); 
data collection procedures and associated errors (4.3); data processing and estimation (4.4); 
and evaluation of estimates (4.5). Most of the material in this chapter will refer to the public 
sector component of the Teacher Demand and Shortage Survey. The main features of the 
private sector component, especially for Round 2, have been described in Chapter 2, in 
connection with the School Survey. 

In Round 1 of SASS there were two versions of the questionnaire for the Teacher Demand 
and Shortage Survey: one for public school districts and one for private schools. The public 
school district version was sent to school districts containing one or more of the public 
schools selected for the School Survey sample, plus a small sample of school districts with no 
schools. The private school version was sent to the same sample of schools that was used for 
the School and School Administrator Surveys. In Round 2, for the private school sector, a 
single questionnaire was used to collect the information for the School and the Teacher 
Demand and Shortage Surveys. 

There were only a few changes in survey content between Rounds 1 and 2, To reduce the 
burden on respondents, a complex accounting-style matrix item that called for data on full- 
time equivalent (FTE) teachers and teaching positions by level and specialty was dropped. 
Items were added in Round 2 to collect information on demand for and shortages of librarians 
and on pension portability. 

The initial sample for the public school sector in Round 2 consisted of 5,424 local education 
agencies. Of these 3.5 percent proved to be out of scope for the survey, and complete 
questionnaires were not obtained for some of the remainder. Further details on response rates 
appear in Tables 3.1 to 3.4. 



4.1 



4.2 Frame development and sampling 



Target populations For the public school sector, the target population consists of U.S. public 
school districts, often called local education agencies (LEAs). An LEA is a local government 
agency administratively responsible for providing public elementary and/or secondary 
instruction and educational support services, operating under a public board of education. 
Some LEAs do not operate schools but hire teachers for schools in other LEAs; an example 
would be a special education program whose teachers are placed in regular schools. Such 
LEAs were included in the target population. LEAs that did not employ any teachers were 
excluded. Each school operated by either of two federal agencies, the Bureau of Indian 
Affairs and the Department of Defense, was treated as a separate LEA. In Minnesota and 
Missouri there were some schools operated by state agencies; these state agencies were not 
included in the LEA target population. 

For the private school sector, the target population for Round 2 of the Teacher Demand and 
Shortage Survey consisted of all U.S. private schools operating in school year 1990-91. The 
design and selection procedures for the private school sample were described in Chapter 2, 
Section 2.2. 

Design and selection of the Round 2 LEA sample As explained in Chapter 2, it would have 
been feasible to sample LEAs as a first step and then sample schools only in the selected 
LEAs. However, a simulation study of alternative designs led to the rejection of this 
approach because it would have substantially increased the sampling variability of school 
estimates (Wright, n.d.). 

The basic design adopted was to include in the sample all LEAs associated with one or more 
sample schools. This design gave every LEA that operated schools a non-zero probability of 
selection, whose value could be calculated for each LEA selected. There were two exceptions 
to the basic approach: 

(1) The Common Core of Data (CCD) frame for the school survey included some LEAs that 
hired teachers but did not operate any schools. The 1988-89 CCD frame used to select the 
school and LEA samples for Round 2 included K352 such LEAs. These LEAs were sorted 
by state, metropolitan area status, 3-digit ZIP code and LEA identification, and then a 1 in 10 
systematic sample was selected. Of the 135 LEAs selected, only 43 were found to be hiring 
teachers during the survey .*cforence period and therefore eligible for inclusion in the Teacher 
Demand and Shortage Survey. 

(2) For three states, Delaware, Nevada and West Virginia, a simulation study prior to Round 
1 showed that the sampling errors of LEA estimates would be quite large even though the 
proposed selection procedures would have included most of the LEAs in those states in the 
sample (Kaufman, 1991, p.27). In both Rounds 1 and 2, therefore, every LEA in those three 
states was treated as a separate stratum for the purpose ol sampling schools, so that all of the 
states' LEAs were included in the sample. 



As indicated in Chapter 2, Section 2.2, a pretest prior to Round 2 suggested that LEAs 
included in the sample for a second round would be less likely to respond than those being 
asked to participate for the first time. For this reason, the school sample overlap was 
controlled at 30 percent. At this level, the expected LEA overlap was 58 percent. 

Frame evaluatio n As already noted in Chapter 2, Section 2.2, a comparison of public school 
estimates with counts from the Public Elementary/Secondary School Universe Survey of the 
Common Core of Data (CCD) series showed that 275 public school districts with only 
elementary schools (Class 1 districts) had not been included in the school frame based on the 
Quality Education Data (QED) list. As a result, the number of LEAs at the national level and 
especially for Nebraska were underestimated from the sample. These schools and hence the 
corresponding LEAs were included in the frame for Round 2. 

4,3 Data collection procedures and associated errors 

Data collection procedures for LEAs in Round 2 In the lr 4 .e Fall of 1990, advance letters 
were mailed to district superintendents for the sample LE, s to alert them to the planned 
SASS data collection activities for their districts. They were informed that a Census Bureau 
representative would be calling them soon to ask that they designate a staff member to take 
responsibility for completing the Teacher Demand and Shortage Survey questionnaire. After 
these calls were made, the survey questionnaires were mailed to the designated persons in 
December 1990 and January 1991. 

About 5 weeks after the first mailing, there was a second mailing to the designated LEA 
respondents who had failed to respond by that time. For those who did not respond to the 
second mailing. Census Bureau field representatives made telephone followups to attempt to 
complete the questionnaires. 

No reinterviews were conducted for the Teacher Demand and Shortage Survey. Frequently, 
more than one person in an LEA provided data for the initial response, which might have 
caused some difficulties in arranging for reinterviews, especially if conducted by telephone. 
As was done for all of the SASS basic surveys, Census Bureau regional office staff reviewed 
a sample of the questionnaires completed by field representatives in their telephone followups 
of nonrcspondents. The procedures were the same as those described for the Sehool Survey 
in Chapter 2, Section 2.3. 

Time required for completion of questionnaire The public school district questionnaires for 
Round 2 included a final question "Not counting interruptions, how long did it take to 
complete this survey?" For questionnaires completed and returned by mail, this item was 
completed by the person responding for the school district; for questionnaires completed in 
followup telephone interviews, the item was completed by the interviewer. The median time 
for completion was 1 hour and 1 5 minutes, with an interquartile range of 90 minutes. About 
5 percent of the districts required more than 5 hours to complete the questionnaire and 1 
percent required more than 10 hours. 
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The State Data Project In Round 1, many LEA respondents to the Teacher Demand and 
Shortage Survey had difficulty completing two matrix items that called for detailed 
information on FTE teachers and positions by teaching level and specialty. Some of the state 
education offices suggested that they might be in a better position than the LEAs to report 
such data to NCES, A feasibility test of this approach was included in the 1990 SASS Pretest 
for Round 2 (Healy, 1990b), 

Initially, 1 1 states were selected to participate in the test. For these states, data for several 
items on the Teacher Demand and Shortage Survey questionnaire were to be collected directly 
from the sample LEAs and also, independently and in computer- readable form, from the state 
education offices. Data from the two sources would be compared, on an item-by-item basis, 
for each of the sample LEAs. 

Eventually, 7 state education offices were able to submit data tapes with the requested LEA 
data. Pretest interviews were completed for 82 of the 96 LEAs in the pretest sample for those 
states. For 2^ of the 38 data elements compared, over 50 percent of the individual LEAs 
reported data that differed by 10 percent or more from the data provided by their state offices. 
On the basis of these results, NCES concluded that it uould not be advisable to try to obtain 
data for LEAs from any states in Round 2 of SASS. It was also decided, as noted earlier in 
this chapter, that the matrix items on FTE teachers and positions would not be included in the 
survey questionnaire for Round 2. 

This outcome does not necessarily rule out the collection from states, in future rounds of 
SASS, of some of the LEA data for the public Teacher Demand and Shortage Survey. In 
November 199\ NCES and the Council of Chief State School Officers convened a Workshop 
on Improving Reliability and Comparability of Staffing Data, the main purpose of which was 
to review the findings from the State Data Project. The participants concluded that state 
education agencies can report district-level data for selected items. The state participants 
expressed a desire to continue to work with NCES to develop a state reporting role in SASS, 
especially for the public Teacher Demand and Shortage Survey (Blank, 1992), However, the 
results of tjie State Data Project demonstrated that further research would be needed to better 
understand the nature of the discrepancies that were observed and to identify the specific 
items which could be adequately reported at the state level. 

Nonresponse error Tables 4. 1 to 4.3 show unweighted and weighted response rates for the 
LEA questionnaire for Rounds 1 and 2 and for the private school Teacher Demand and 
Shortage Survey questionnaire for Round 1 only (in Round 2 it was combined with the 
School Survey questionnaire, for which response rates are shown in Chapter 2, Tables 2.5 to 
2.7), LEAs and private schools that were not operating in the school year of reference for the 
survey or failed to meet the definition for other reasons are excluded from the bases of the 
response rates. Table 4.4 shows, for LEAs in Round 2, the percentage of the initial sample 
that was excluded for such reasons. As explained below, under measurement errors, a few 
questionnaires that were initially counted as respondents were subsequently excluded from the 
survey estimates because of reporting errors that could not readily be corrected. 

4.4 
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As shown in Table 4.1, Round 1 response rates for LEAs were about 25 percentage points 
higher than those for the private school Teacher Demand and Shortage Survey questionnaire. 
For Round 2, the weighted LEA response rate was 93.6 percent, compared with 83.9 percent 
for the private School Survey questionnaire (which included the teacher demand and shortage 
items), so the gap between the public and private school sec.^s has narrowed somewhat. For 
both sectors the response rates increased between Rounds 1 and 2, in the face of a predicted 
decline as a result of expected lower response in Round 2 for LEAs and schools that had 
already been included in Round 1. Actually, in Round 2 the response rate for overlap LEAs 
(those that had been in the sample in Round 1) was about the same as the rate for nonoverlap 
LEAs. 

The higher response rates in Round 2 may have been due in part to the elimination of the 
troublesome matrix items on FTE teachers and positions. For private schools in Round 1, 
when separate questionnaires were used, the response to the School Survey questionnaire was 
higher than the response to the Teacher Demand and Shortage questionnaire (78.6 percent 
versus 66.0 percent, weighted). Combining these two questionnaires for Round 2 may also 
have contributed to the higher response rates. 

There was considerable variation in response rates within the public and private sectors. In 
Round 1, in the public sector, 17 states had weighted LEA response rates of 95 percent or 
better and only 1, Connecticut, was below 80 percent. In Round 2 there were 25 states with 
weighted response rates of 95 percent or oetter and, as before, only Connecticut was below 80 
percent (Table 4.2). In the private sector, the weighted response rates to the Round 1 Teacher 
Demand and Shortage questionnaire by association group varied from a low of 38.8 percent to 
a high of 91.7 percent, with only 4 of 14 groups having response rates of 80 percent or higher 
(Table 4.3). For Round 2, the teacher demand and shortage items were included in the 
private school questionnaire, so the applicable response rates are those shown for schools in 
Table 2.7b, Chapter 2. 

Data from published summaries of unweighted item response rates for the Teacher Demand 
and Shortage Survey are shown in Table 4.5. For Round 2, data are shown only for the 
LEAs, because the available data do not make it possible to distinguish teacher demand and 
shortage items from other items on the School Survey questionnaire for private schools. Most 
of the items with low response rates in Round 1 were associated with the two matrix items on 
FTE teachers and positions by field of assignment. Because of the low response rates and 
other indications of poor quality, no data based on these two questionnaire items were 
published and they were not included in the survey data files. In Round 2, which did not 
include these items, the overall level of item response was substantially improved. 

Measurement errors associated with data collection For Round 1, at the time the weights 
were being applied to LEA data, it was observed that for some LEAs the numbers of students 
or teachers were much higher or lower than expected on the basis of prior year data for the 
same LEAs. A listing was prepared of all LEAs for which (a) reported counts of students or 
teachers differed by 35 percent or more from expected counts, or (b) the student/teacher ratio 
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was greater than 35 or less than 10. Reviewers of the listed cases identified 290 LEAs for 
which counts of students or teachers appeared to be incorrect, including 46 LEAs which 
appeared to have reported data for sample schools only, rather than the entire district. 

Further review of the questionnaires for the 290 LEAs identified: 

• Several cases where LEAs had merged and one case which was out of scope because 
all teachers and students were preschool. 

• Thirty-three LEAs for which the entire questionnaire had been completed for a single 
school, rather than the entire LEA. (This is the converse of the situation discussed in 
Ch. Dter 2, Section 2.5, where it was found that some Round 1 School Survey 
questionnaires had been completed for the entire district rather than the specified 
sample school.) These sample LEAs were reclassified as noninterview cases, reducing 
the response rate for the public Teacher Demand and Shortage Survey by slightly more 
than 1 percent. 

As a result of these findings and the actions that were taken, the sample weights had to be 
recalculated. For the LEAs that were not reclassified as noninterviews, values that were 
clearly incorrect were replaced by imputed values, based primarily on other items from the 
same questionnaire or data on the sample file (Fondelier, 1990). 

The 1990 Pretest of the Round 2 questionnaire for the public Teacher Demand and Shortage 
Survey identified some potential reporting problems (Healy, 1990a). One of these had to do 
with the categories used for grade level in questions about staffing and enrollment: pre- 
kindergarten, kindergarten, 1-6 and 7-12. Some districts use other grade structures for their 
school data, for example, kindergarten to 5, 6-8 and 9-12. In the pretest, 12 of 283 LEAs 
handled this problem by crossing out the grade level categories on their questionnaires and 
writing in new ones. This observation suggested the need for a clerical check of completed 
questionnaires to make adjustments in such cases. 

Numerous instances of incomplete or incorrect reporting in the Pretest were observed with the 
matrix items that called for data on FTE teachers and positions by grade level and specialty. 
As noted earlier, these two items were dropped from the final questionnaire for Round 2. 

A general observation from this review was that the quality of questionnaires returned by mail 
appeared to surpass that of the ones that had been completed by telephone follow-ups. The 
reviewer noted that the questionnaire is difficult to complete by telephone and that some of 
these cases were "quasi refusals", with respondents reluctantly providing minimal data. 

Indications of response error in Round 2 of the public Teacher Demand and Shortage Survey 
come from several sources: an early review of unedited and edited questionnaires received 
from the field (Healy and Pasqualucci, 1991); memoranda submitted by Census regional 
offices following the completion of SASS data collection for Round 2 (Bureau of the Census, 
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1991 a); a review of the forms completed by regional office staff for their reviews of 
questionnaires obtained by Census field representatives in telephone followups of mail 
nonrespondents (Pasqualucci, 1991); and a review of pre-edit reject rates, edit change tallies 
and post-edit item response rates (Jenkins, 1992a). 

Types of response errors mentioned in these sources included: 

As noted in connection with the 1990 Pretest, a few respondents changed the grade 
level categories for reporting enrollments and staff, for example from 1-6 and 7-12 to 
1-8 and 9-12. This was observed in the early review of incoming questionnaires and a 
procedure was added to the clerical edit to check for this type of alteration. 

As in Round 1, many respondents failed to record decimal entries in the manner 
intended for the items relating to FTE staff and graduation requirements. In most 
instances, such reporting errors can be detected and corrected either in clerical edits or 
as a result of consistency checks included in computer edits. 

Many respondents failed to observe skip instructions and unnecessarily completed an 
item on the overall range of base year teacher salaries in the district. This item was 
intended only for respondents who could not provide separate ranges for different 
levels of qualification. Failure to skip did not affect the accuracy of the data for these 
items. 

For two topics, one relating to FTE teachers and one to FTE librarians and media 
specialists, there were frequent discrepancies between component items and overall 
totals. 

In most instances, it was possible to detect response errors of these kinds in the clerical and 
computer edits and to substitute correct or at least more nearly correct values on the basis of 
other information on the questionnaire or in the sample file, or from telephone contacts in the 
early stages of processing. 

Comments from Census regional offices are also available for the Round 3 SASS Pretest, 
conducted during the first half of 1992. Those relating specifically to the public Teacher 
Demand and Shortage questionnaires dealt primarily with issues of respondent burden 
resulting from complexity and requirements to complete matrix-style items. Understanding of 
the FTE concept continues to be considered a problem; one Census field representative was 
quoted as saying "I have yet to find one respondent whom I feel really understands the 
concept of FTE" (Bureau of the Census, 1992). 
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4.4 Data processing and estimation 



Data processing procedures The sequence and nature of the data processing procedures for 
the Round 2 LEA Teacher Demand and Shortage Survey were essentially the same as for the 
other basic SASS surveys. Processing procedures for the School Survey are described in 
Chapter 2, Section 2.4 and the sequence and location of processing steps are shown in Exhibit 
2.1. 

Imputation in Round 2 Items that were missing or failed range or consistency checks were 
imputed at various stages of processing in both Rounds 1 and 2. In the first phase of 
computer imputation, values were imputed by the application of logical rules to other data for 
the same LEA. Data sources for imputation of missing items included other items on the 
same questionnaire, LEA data from the Common Core of Data frame and, if all of the schools 
in an LEA had been included in the sample, data from the School Survey questionnaires for 
those schools. For items that could not be imputed in the first phase, a hot deck procedure 
based on responses for other LEAs with similar characteristics was used. Imputation flags 
were assigned in the same manner as described for schools in Chapter 2, Section 2.4. 

Weighting The overall weights applied to LEA data for Round 2, like those used for the 
School and School Administrator Surveys, were the product of four factors: a basic sampling 
weight; a sampling adjustment factor; an LEA nonresponse adjustment factor; and a frame 
ratio adjustment factor. 

Because of the method used to select the sample of LEAs, calculation of the basic sampling 
weights for LEAs was somewhat more complex than it was for schools, school administrators 
and teachers. For LEAs with no schools, the basic weight was 10, because 1 in 10 of these 
LEAs had been selected. For LEAs in Delaware, Nevada and West Virginia, the basic weight 
was 1, because all LEAs in those states were selected. For the remaining LEAs with schools, 
the basic sampling weight for an LEA was the inverse of one minus the product of the 
probability of no schools being selected from each of six strata that were used in selecting the 
sample of schools. 

The other three factors used in the calculation of the overall weights were similar to those 
described for schools in Chapter 2, Section 2.4. Sampling adjustment factors were needed to 
account for unusual factors affecting an LEA's probability of selection, such a merger with 
another LEA, a split into two or more LEAs or duplicate listings of the LEA in the sampling 
frame. For the nonresponse and frame ratio adjustment factors, the definitions of adjustment 
cells and the collapsing rules were similar to those used for schools. 

Variance estimation As described in Chapter 2, Section 2.4, a balanced half-sample 
replication (BHR) procedure has been used to estimate sampling errors for all SASS surveys. 
Replicate weights for use in estimating sampling errors by this procedure are included in all 5fc x* 
microdata files, so that users of these files can estimate sampling errors for items of interest to 
them. The majority of LEAs were selected through the sample of schools, and the 48 half 
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sample replicates for these LEAs were formed using the corresponding school replicates. An 
LEA was placed into an LEA replicate if any of the sample schools associated with the LEA 
had been included in the corresponding school replicate. LEAs that had been selected with 
certainty were included in all replicates, and a separate procedure was used to assign sample 
LEAs with no schools to replicates. 

The BHR variance estimation procedure assumes sampling with replacement, whereas 
sampling without replacement is used for all of the SASS surveys. Violation of the 
assumption leads to BHR overestimates of the true variances, but the effects should be small 
unless the sampling fractions are quite large (Kaufman, 1991). For some states, the 
proportion of LEAs sampled is large, so large overestimates of variance are more likely to 
occur for the public Teacher Demand and Shortage Survey estimates for these states. 

The public Teacher Demand and Shortage Survey is different from the other SASS surveys in 
that its unit for data collection and analysis, the LEA, is an aggregate of the sampling units, 
which are schools. This sample design leads to possible violation of a second assumption that 
is implicit in the BHR method of variance estimation, namely that the true variance is 
inversely proportional to the sample size. Kaufman (1992, 1993, 1994) has undertaken a 
series of simulation experiments to determine the extent of bias for the current variance 
estimation procedures and to evaluate some alternatives. One finding has been that the BHR 
variance estimates for the public Teacher Demand and Shortage Survey for Rounds 1 and 2 
have been substantial overestimates. The extent of overestimation varies by state; for some 
states confidence intervals based on estimates plus or minus one standard error covered the 
corresponding population values more than 90 percent of the time (if the variance estimates 
were unbiased, this should happen about 68 percent of the time). 

One of the simulation experiments evaluated two different weighting procedures; the one 
currently in use, in which the sampling weight for each LEA is the inverse of its selection 
probability, and an alternative "expected hits" weighting procedure, in v/hich the weights are 
based on the selection probabilities of the sample schools within the selected LEAs. The 
alternative weighting procedure satisfies the second of the two assumptions of the BHR 
method of variance estimation and therefore should produce unbiased estimates of variances, 
using the BHR method, if sampling were done with replacement. This proved to be the case; 
however, for averages and ratios, the estimates based on expected hit weights had larger 
variances than those based on probability weights, so a change in the present weighting 
scheme was not deemed advisable. 

The study also evaluated two different methods of variance estimation: the BHR and the 
bootstrap methods. In some respects the bootstrap method appears to work well, and it is 
being considered for use in future rounds of SASS. If this is done, it will still be possible for 
users to compute variances using any BHR program without modification (Kaufman, 1993, 
1994). 
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4*5 Evaluation of estimates 



Round 1 As noted in Section 4.2, some school districts in Nebraska that had only elementary 
schools were found to have been omitted from the Round 1 sampling frame based on the 
Quality Education Data list. Consequently, estimated numbers of LEAs, schools, teachers and 
students for Nebraska were low in Round 1. 

Round 2: Post processing edit When the initial set of weighted estimates was available, the 
counts of school districts by state were compared with the 1988-89 CCD, which had served as 
the sampling frame for Round 2, and the numbers of teachers and students were compared 
with the corresponding data from the 1990-91 CCD, covering the same reference year. Eight 
states had SASS estimates of teachers or students that exceeded the CCD count by 15 percent 
or more. Examination showed that these overestimates resulted from the erroneous inclusion 
in SASS of school districts that were supervisory unions or other districts that did not hire 
teachers. All such districts were reclassified as out of scope and their data eliminated from 
the estimates. 

Round 2: Evaluation of published estimates The final estimates, reflecting changes made in 
the post-processing edit, were again compared with CCD data by state. The estimated 
numbers of LEAs from SASS were compared with two counts from the 1988-89 CCD: the 
total number of LEAs and the number of regular LEAs. These comparisons were complicated 
by the fact that the character and definitions of LEAs vary by state: some of the LEAs not 
counted as regular in the CCD do hire teachers and were therefore eligible to be included in 
SASS. 

For 14 states, the SASS estimate of LEAs differed from the CCD count of regular or total 
LEAs by 15 percent or more. Estimates for these states were reviewed in detail and in some 
instances state or local education agencies were called to obtain information about the nature 
of non-regular LEAs. For each of the 14 states, a CCD count of LEAs was determined that 
came as close as possible to meeting the SASS definition for eligible LEAs, and the SASS 
estimate was compared with that count. Based on this comparison, the SASS estimate was 
within 10 percent of the CCD count in 10 of the states and within 15 percent in the remaining 
4 states. 

Nationally, the SASS estimate of public school teachers, as reported by the LEAs, was 5.9 
percent below the CCD count. There were 4 states whose SASS estimates were more than 1 5 
percent below the CCD counts: Maryland, Michigan, New Mexico and Texas, The largest 
underestimate was for New Mexico, which was 18,6 percent below the CCD count. For 
enrollment, the U.S. estimate was 2.7 percent below the CCD count, and there were 3 states — 
Michigan, Nevada and New Mexico — for which SASS estimates of enrollment were from 10 
to 15 percent below the CCD counts. For all other states, SASS estimates were within 10 
percent of the CCD figures. Details by state for all of these comparisons are provided in 
Chapter Xll of the Round 2 Data File User's Manual, 
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Table 4.1 



Teacher Demand and Shortage Survey Response Rates 





Round 1 (1988) 


Round 2 (1991) 


Unweighted 


Weighted 


Unweighted 


Weighted 


Public (LB As) 


89.4 


90.4 


93.7 


93.5 


Private 


67.9 


66.0 


84.8*-' 





Notes: 



I. Response rates for the combined School and Teacher Demand and Shortage Surveys. 



Sources: 

Round 1 Unweighted: Kindel (1989). 

Round 1 Weighted: NCES (1991c). 

Round 2: (Jruber, Rohr and Fondelier (1993). 
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Table 4.2 Teacher Demand and Shortage Survey Weighted Response Rates for Public 
Districts by State 





Response Rate 




Response Rate 


State 


Round 1 


Round 2 


State 


Round 1 


Round 2 


Alabama 


97.0 


96.3 


Montana 


88.6 


95.1 


Alaska 


100.0 


96.2 


Nebraska 


91.8 


97.3 


Arizona 


92.1 


90.4 


Nevada 


100.0 


100.0 


Arkansas 


95.9 


91.3 


New Hampshire 


84.1 


92.9 


California 


90.4 


91.3 


New Jersey 


83.9 


86.3 


Colorado 


95.5 


98.2 


New Mexico 


87.9 


95.0 


Connecticut 


61.1 


77.0 


New York 


91.0 


95.7 


Delaware 


94.7 


100.0 


North Carolina 


SS.l 


94.0 


District of CoIl )ia 


100.0 


100.0 


North Dakota 


93.6 


94.4 


Florida 


92.5 


92.0 


Ohio 


98.2 


89.4 


Georgia 


81.3 


92.3 


Oklahoma 


97.7 


98.5 


Hawaii 


100.0 


100.0 


Oregon 


98.5 


91.2 


Idaho 


97.3 


95.5 


Pennsylvania 


S4.2 


94.4 


Illinois 


93.4 


91.8 


Rhode Island 


100.0 


91.9 


Indiana 


97.6 


95.8 


South Carolina 


83.9 


92.8 


Iowa 


89.S 


98.4 


South Dakota 


97.4 


98.2 


Kansas 


85.4 


99.6 


Tennessee 


91.6 


100.0 


Kentucky 


86.5 


92.3 


Texas 


90.1 


95.2 


Louisiana 


91.7 


90.1 


Utah 


97.4 


96.0 


Maine 


88.1 


92.0 


Vermont 


99.2 


86.9 


Maryland 


87.9 


87.5 


Virginia 


90.8 


90.7 


Massachusetts 


83.5 


94.1 


TYdZmlllgltJII 


81.4 


97.0 


Michigan 


96.6 


90.2 


West Virginia 


87.3 


98.2 


Minnesota 


87.3 


92.1 


Wisconsin 


85.0 


96.3 


Mississippi 


93.0 


96.7 


Wyoming 


92.5 


96.1 


Missouri 


92.0 


93.8 


TOTAL 


90.8 


93.5 



Sources: NCES (1991c) and (Jruber, Rohr and Fondelier (1993). 
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Table 4.3 Teacher Demand and Shortage Survey Weighted Response Rates for Private 
Schools by Association Group: Round 1 



Association Group 


Response Rate 
(Percent) 


Total 


66.0 


Area Sample 


49.0 


Association of Military Colleges and Schools - US 


91.7 


Catholic 


84.3 


Friends 


77.8 


Episcopal 


65.6 


Jewish 


53.2 


Lutheran 


83.2 


Seventh-day Adventists 


71.7 


Christian Schools International 


88.5 


American Association of Christian Schools 


38.8 


National Association of Private Schools for Exceptional 
Children 


65.3 


American Montessori Society 


73.6 


National Association of Independent Schools 


63.7 


Other 


54.5 



Source: NCES (1991c). 
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Table 4.5 Teacher Demand and Shortage Survey Unweighted Item Response Rates 



Sector 


Range of Item 
Response Rates 
(Percent) 


Percent of Items with 
Response Rates: 


> 90% 


< 75% 




- - - a . .. ..^ . : ■ ■ : y : : . .. ... .. .: 

> s . ^ , ■ <" ... 




Public (LEAs) 


40- 100 


74 


12 


Private 


16 - 100 


70 


18 




- -, :--■■--.; -.-.--,--- :-> -.- - - -- --. - - -- 


•: : :•. .•• . ■ . ■ 
- - - ,- ... - . - y . : .y, , . ... .. 

........... .■ . . . 

- ;. ->: : :w : >■ >■ ; .... 


Public (LEAs) 


85 - 100 


90 


0 



Sources: 

NCES (1991c). 

Gruber, Rohr and Fondelier (1993). 
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CHAPTER 5 



THE TEACHER SURVEY 



5*1 Introduction 

In the Teacher Survey, data are collected from a sample of classroom teachers in each of the 
public and private schools that were included in the sample for the School Survey. Data for 
schools and teachers are collected for the same school year (1987-88 for Round 1 of SASS 
and 1990-91 for Round 2). In the following school year, data are collected in the Teacher 
Followup Survey for all sample teachers who have left the sample school and for a subsampie 
of those still teaching in the same school. 

This chapter follows our standard structure for presenting information on each of the 
component surveys of SASS. The survey design and procedures are described for the Round 
2 Teacher Survey, along with information on significant changes between Rounds 1 and 2. 
Information on the quality of data is presented for both rounds. The four sections which 
follow cover the main phases of the survey: frame development and sampling (5.2); data 
collection procedures and associated errors (5.3); data processing and estimation (5.4); and 
evaluation of estimates (5.5). The Teacher Survey design and procedures for the public and 
private school sectors are quite similar, so they will be described as a single survey, with 
differences noted where they exist. 

The topics covered in the Teacher Survey questionnaires for Rounds 1 and 2 were similar. 
They include: current teaching status and work load; past teaching experience; education and 
training; perceptions and attitudes toward teaching; incentives and compensation; demographic 
characteristics; and tracing information needed to locate teachers included in the Teacher 
Followup Survey. 

There were some changes in specific items, especially in the questionnaire section on 
perceptions and attitudes toward teaching. In an initial item asking teachers about their 
agreement or disagreement with each of a series of statements relating to their teaching 
environment, the number of such statements was reduced from 23 in Round 1 to 14 in Round 
2. Conversely, for an item asking teachers to evaluate the relative seriousness, in their school, 
of various kinds of problems, the number of categories was increased from 13 to 22. A new 
item asking teachers to rank the relative importance of various educational goals was added in 
Round 2. 

In the questionnaire section on teacher training, items on membership in professional 
organizations and participation in teacher induction programs (assistance to new teachers by 
mentor or master teachers) were added in Round 2. Under the heading of incentives and 
compensation, a Round I item on incentives asked, with respect to each of several possible 
kinds of pay incentives, whether teachers favored them and whether they were receiving them. 
In Round 2, teachers were asked only to report which ones they were receiving. 
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There were separate questionnaires for public and private school teachers in both rounds, but 
the contents were nearly identical. In Round 2, different lists of the organizations endorsing 
the two surveys were presented on the front page of the public and private school versions. 
One item, asking whether the teacher was working at the school on a contributed service basis 
(less than full salary or no salary), was used only for private school teachers in Round 2. 

The initial samples for the Round 2 Teacher Survey consisted of 56,051 public school 
teachers and 9,166 private school teachers. Of these, 7.1 percent of the public school teachers 
and 1 2.4 of the private school teachers were later found to be ineligible for the survey, and 
completed questionnaires were not obtained for all of those who were eligible. Further details 
on sample sizes and response rates are given in Tables 5.1 to 5.5. 

5.2 Frame development a;.d sampling 

The target population The target population for the Round 2 Teacher Survey consisted of 
regular full-time and part-time teachers whose primary assignment was teaching in 
kindergarten or any of grades 1 to 12 during school year 1990-91. Also included were long- 
term substitutes who were filling the role of a regular teacher on a long-term basis and 
itinerant teachers (those teaching regularly in more than one school). 

If a school was considered ineligible for the School Survey (see Chapter 2, Section 2.2), that 
school's teachers were ineligible for the Teacher Survey. Also excluded from the Teacher 
Survey target population were: short-term substitutes, teacher's aides, student teachers, 
administrators, and other non-teaching professional and support staff, as well as teachers no 
longer working at the school from which they had been selected. 

Ineligible persons could be screened out at three stages. The instructions called for them to 
be excluded from the teacher lists requested from the sample schools for use in selecting a 
sample. Any persons who had been incorrectly included on a school's teacher list should 
have been identified by the first item on the Teacher Survey questionnaires, which asked for 
the respondent's main assignment at the school during the current school year. Respondents 
in categories, such as student teacher, that made them ineligible for the Teacher Survey were 
instructed to mail back their questionnaires without completing the remaining items. Finally, 
if a full questionnaire was returned for an ineligible person, it would normally be classified as 
out-of-scope in the interview status edit (see Section 5.4). 

Sample design objectives and considerations The goals that guided the design of the Teacher 
Survey in Round 2 of SASS were: 

( 1 ) Provide estimates of acceptable precision for specified domains of analysis. These 
domains included: (1) experienced and new teachers in public schools by state and level 
(elementary, secondary or combined) and in private schools by association category, level and 
Census region; (2) bilingual/ESL teachers for California, Florida, Illinois, New York, Texas, 
and all other states as a group; (3) Asian or Pacific Islander teachers; and (4) American 
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Indian or Alaskan native teachers. To meet this goal required oversampling (relative to other 
groups) of new teachers in private schools and of each of the last three domains. The last 
two domains were new in Round 2; the first two were targeted in both rounds. New teachers 
were those in their first, second or third year of teaching; all others were classified as 
experienced. Bilingual/ESL teachers were those who (1) were using native language to 
varying degrees to instruct students with limited English proficiency or (2) providing intensive 
instruction in English to students with limited English proficiency. 

(2) Place limits on the number of teachers selected from each sample school. At least one 
teacher was to be selected from every sample school. The number to be selected from a 
school was not allowed to exceed either 20 or twice the average allocation of teachers per 
school for the stratum in which the school was included, if that value was less than 20. 

(3) Make the sample approximately self weighting within each of the main analytical 
domains. This goal was accomplished by using within school sampling fractions for selection 
of teachers that, when multiplied by the schools' selection probabilities, would produce 
approximately the same teacher selection probabilities for all schools in a domain or stratum, 
for example, public elementary schools in a state. It was expected that this design would be 
close to optimum with regard to the sampling errors of estimates for each domain. 

(4) Ensure that the target sample sizes would be achieved for each analytical domain. In 
advance of sample selection, there were no reliable estimates available for the numbers of 
bilingual/ESL, Asian and Pacific Islander, and American Indian teachers. The goal was 
achieved by using initial sampling fractions expected to produce larger than needed samples 
for each domain, randomly assigning the selected samples into 101 equal size "reduction 
groups", and then deleting reduction groups as needed to achieve the target sample size in 
each domain. 

The teacher sampling frame for Round 2 In early October 1990, advance letters and forms 
for listing teachers were mailed from the Census Bureau's Jeffersonville processing facility to 
all sample schools. The listing forms contained instructions for listing eligible teachers. For 
each teacher, the school was asked to report first and last name, teaching experience, race, 
participation in ESL/bilingual programs, and teaching specialty by level The forms were to 
be mailed back to Jeffersonville. 

Starting in mid-October, Census Bureau field representatives telephoned sample schools which 
had not yet returned their teacher listing forms. They asked these schools to (in order of 
preference): mail in their completed forms; mail in a list of their teachers; give the list of 
teachers over the telephone; or, if the school objected to providing a complete list of 
teachers, select a sample of teachers as instructed by the field representative and provide 
information for the selected teachers over the telephone. 

Some schools that selected their own samples objected to providing the names of sample 
teachers, *jo they were asked to label the ones they had selected as Tl, T2, T3, etc. The 
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questionnaires for these teachers were subsequently mailed to the contact person for the 
sample school labelled with these identifiers (Bureau of the Census, 1990). This procedure 
was used for about 1 percent of the public schools and about 3 percent of the private schools. 

Frame evaluation Because the teacher listings were obtained near the beginning of school 
year 1990-91, only those teachers who were on the school's rolls at that time would have 
been listed and eligible for sampling. A school's roll of teachers might have included some 
teachers who were on leave during the initial part of the school year; however, the 
instructions for the teacher listing forms did not specify whether or not teachers on leave 
should be included. 

In Round 2 of SASS, teacher lists or samples were not provided by 5 percent of the eligible 
public schools and 10 percent of the eligible private schools that responded in the School 
Survey. No teachers were selected for these schools (Gruber, Rohr and Fondelier, 1993, 
Chapter VI). The corresponding figures for Round 1 were 4 percent for public schools and 
12 percent for private schools (Kaufman, 1991, p.37). 

The count of eligible teachers from the school questionnaire was not always the same as the 
number of teachers recorded on the listing form for the same school. In Round I, in the 
average state, there were 5 percent fewer teachers recorded on the listing forms than were 
reported on the school questionnaires (Kaufman, 1991, p.67). If school questionnaire counts 
were correct, the teacher weights used in Round 1 would have resulted in underestimates of 
the number of teachers. As explained in Section 5.4, the teacher weights for Round 2 
included a factor to benchmark the estimates to the school counts of teachers. The values of 
these factors by weighting cell indicate that, on the average, fewer teachers were recorded on 
the listing forms than were reported on the school questionnaires. 

To explore the reasons for these discrepancies and to determine which counts are more 
accurate, a Teacher Listing Validation Study was undertaken in school year 1992-93 (Royce 
and Schreiner, 1994). That study has verified that some teachers, especially those working 
part-time, arc missed by the teacher listing operation, but it does not provide a basis for 
estimating what proportion are missed. Further details are given in Chapter 7, Section 7.3. 

Sample selection for Round 2 The main steps in the selection of the sample of teachers for 
Round 2 were as follows: 

(1) Determine the total number of teachers to be selected from the list provided by each 
sample school. These numbers were chosen according to a formula that satisfied two 
requirements: (a) make the product of the school's selection probability and the 
sampling fraction for teachers within the school constant within each stratum, and (b) 
make the average number of teachers selected from schools in that stratum equal to a 
target number established for schools of that sector (public or private) and level 
(elementary, secondary or combined). 
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(2) Allocate the sample for each school between experienced and new teachers. In public 
schools, they were allocated in proportion to the number of teachers of each type, as 
determined from the teacher listing forms. In private schools, new teachers were 
oversampled by a factor of L8 (in Round 1 a factor of L6 had been used), in order to 
ensure a sufficiently large sample from this category in both the Teacher and Teacher 
Followup Surveys. 

(3) For each school, sort the teachers into five groups, as follows: (a) Asian or Pacific 
Islander teachers, (b) American Indian or Alaskan native teachers, (c) bilingual/ESL 
teachers, (d) new teachers, and (c) experienced teachers. Teachers falling in more th^n 
one of these categories were to be placed in the first one listed. Within each of the-^e 
groups, sort teachers by primary field of ieach : ,ng, as recorded on the teacher listing, 
form for the school. 

(4) Within each school and group, select a systematic (every nth) sample of teachers at tre 
rate determined for that group as a result of steps (1) and (2). 

(5) For the first three groups in step (3), randomly assign the imple teachers to 101 
subsamples (reduction groups). For each of these three groups eliminate subsamples as 
needed to produce a sample of approximately the size specified for that group. 

The resulting teacher sample sizes were as follows: 



Asian or Pacific Islander 1,51 1 

American or Alaskan native 1,529 

Bilingual/ESL 2,121 

New 7,972 

Experienced 52,084 

Total 65,217 



5.3 Data collection procedures and associated errors 

Data collection procedures for Round 2 Questionnaires were mailed to the sample teachers at 
their schools during January and February of 1991* In February and March, a second 
questionnaire was sent to each sample teacher who had not yet responded, In March, 
telephone followups to nonrespondents were initiated by Census Bureau field representatives, 
calling either from their homes or from the Census Bureau* s regional offices. Telephone 
followups continued through June 1991. 

The field representatives were instructed to call teachers at the schools to attempt interviews 
during non-teaching hours, for example, 8:00 to 9:00 a,m, and 3:00 to 5:00 p.m. For teachers 
unable to be interviewed at those times, they were to ask whether it would be possible to 
conduct the interview during a planning or free period. 
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Special procedures were 1 ecessary for schools whose sample teachers had been identified only 
by alphanumeric indicators (Tl, T2, etc.)- For these schools, the initial and followup 
questionnaires were mailed to the principal or other contact person at the school for 
distribution to the sample teachers. When telephone followups were necessary, the Census 
field representative was instructed to call the principal or other contact person and ask that the 
teacher(s) who had not responded be requested to call the field representative in order to 
complete their interviews. 

Changes between Rounds 1 and 2 In Round 1, because of the relatively large number of 
teachers who did not mail back their questionnaires, the telephone followups were conducted 
only for a sample of the nonrespondents. In Round 2, all nonrespondents were followed up 
by telephone. 

In Round 1, a school coordinator was appointed for each school to assist with the distribution 
of questionnaires for the Teacher Survey and the followup of nonresponding teachers. About 
10 days after the initial mailing of questionnaires to the sample teachers, a letter was sent to 
each coordinator, listing the sample teachers and asking the coordinator to remind them to 
complete and return their questionnaires. About 6 weeks after the first mailing, replacement 
questionnaires for the nonresponding teachers were sent to the coordinator in a package. The 
coordinator was also contacted by telephone and asked to distribute the questionnaires to the 
teachers and encourage them to complete and return their forms (NCES, 1992). 

The school coordinator procedure used in Round 1 was based on the findings from a test of 
alternative methods of using school coordinators, which had demonstrated that response rates 
were higher for schools with coordinators, but that payment versus nonpayment of 
coordinators had no measurable effect on the results (Schwanz, 1987; Kaufman, 1988). The 
procedure was dropped in Round 2 in order to protect, to the greatest degree possible, the 
identity of the sample teachers in each school and, hence, the confidentiality of the data they 
were providing in the survey. 

Census Bureau field staff comments on collection procedures Following completion of field 
work for Round 2 of SASS and again after the completion of the 1992 pretest for Round 3, 
each Census regional office was asked to submit its comments and suggestions on how the 
survey instruments end collection procedures could be improved. Among the most frequent 
suggestions relating to the Teacher Survey were the following (Bureau of the Census, 1991a, 
1992): 

• Distribute the questionnaires earlier in the school year, so that teachers and school 
administrators would not be asked to complete them during the period leading up to 
graduation, which is one of their busiest times of year. 

• Develop a procedure for conducting telephone followup interviews with teachers by 
calling them at their homes. Completing telephone interviews during school hours 
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proved difficult because of the length of the questionnaire, the sensitivity of some of 
the items, and limited access to telephones in many schools, 

• Make the cover oage of the Teacher Survey questionnaire "friendlier" by including 
more information specifically addressed to the teacher, such as the letter from the 
Commissioner of NCES, 

A procedure for conducting telephone followup interviews with teachers at their homes was 
tested in the pretest for Round 3, as described below, 

Test of new followup procedure In Rounds 1 and 2, Census field representatives often had 
difficulty contacting and conducting telephone interviews with teachers at school. Many 
teachers do not have easy access to telephones or are not able to use them for the time 
necessary to complete an interview. In a pretest for Round 3, conducted early in 1992, a 
postcard was sent to each sample teacher who had not responded within about two weeks of 
the second mailing. The message included the following: 

If we do not receive your completed questionnaire by mail within two weeks, we will 
contact you by telephone at your school to collect this information. If you prefer to be 
contacted at home, please provide your home telephone number on the attached 
postcard and reiurn it in the next few days. 

Teachers who supplied their telephone numbers were contacted for interviews at their homes 
(Ferrell, 1992). Only a small proportion of teachers returned the postcards; therefore, this 
procedure has not been formally adopted for nonresponse followups in Round 3. 

Time required for completion of questionnaire The teacher questionnaires for Round 2 
included a final question "Not counting interruptions, how long did it take to complete this 
survey?" For questionnaires completed and returned by mail, this item was completed by the 
responding teacher; for questionnaires completed in followup telephone interviews, the item 
was completed by the interviewer. The median time for completion was 45 minutes for 
public school teachers and 40 minutes for private school teachers, with interquartile ranges of 
30 minutes for both groups. About 99 percent of all teachers completed the questionnaire in 
less than 2 hours* 

Supervision and quality assurance As in most of the other SASS surveys, the two primary 
methods of controlling the quality of the data collection operations were regional office 
reviews of the questionnaires completed by telephone for nonrespondents and reinterviews of 
respondents for a sample of completed questionnaires. 

In the regional office reviews, the first four Teacher Survey questionnaires completed by each 
field representative were checked. If the total number of errors (including omissions) for the 
four questionnaires was ten or more, additional questionnaires were reviewed until there were 
four consecutive questionnaires with a total of fewer than ten errors. For the other three basic 
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surveys, the standard used for this review was fewer than ten total errors in two 
questionnaires (Gruber, Rohr and Fondelier, 1993, Chapter VII). 

For both rounds of SASS, re- jrviews were attempted for about 1,100 teachers, or about 1.65 
percent of the total. They we.e successfully completed for about 75 percent of the eligible 
cases in Round 1 and 83 percent in Round 2, All reinterviews of teachers in both rounds 
were conducted by telephone. Findings from reinterviews are presented below, under the 
heading "Measurement error, findings from reinterviews". 

Nonresponse error For the Teacher Survey, unit nonresponse could occur for two reasons: 
(1) a sample school failed to provide a list of teachers for use in selecting a sample (or, 
alternatively, to select a sample of teachers itself), or (2) an acceptable questionnaire was not 
obtained for a sample teacher. We will refer to these two sources of nonresponse as school 
nonresponse and teacher nonresponse. The product of the two types of response (the 
complement of non/esponse) is an indicator of overall teacher response rates. Table 5.1 
shows these overall response rates, for public and private schools, for Rounds 1 and 2. The 
school response rates (proportion of schools providing teacher lists) are unweighted; the 
teacher response rates are weighted. 

Table 5.1 shows that there were higher response rates, at both stages and in both rounds, for 
public schools. Overall response rates for public schools were 13 percentage points higher 
than those for private schools in Round 1 and 1 1 percentage points higher in Round 2. 
Mainly due to higher teacher response rates in Round 2, the combined rates were higher for 
both public schools (3 percentage points) and private schools (5 percentage points). Based on 
the results of the Round 1 pretest experiment with coordinators, one might have predicted 
lower teacher response rates in Round 2, when no coordinators were used, but the reverse 
occurred. 

Table 5.2 shows unweighted and weighted teacher response rates for both rounds, for public 
and private schools. Weighted response rates take into account the probabilities with which 
schools in various strata were selected and are therefore a better indicator of the potential 
effects of nonresponse on the survey estimates. Table 5.3 shows weighted teacher response 
rates for public schools by state for Rounds 1 and 2, For Round 1 the rates ranged from a 
low of 68.6 percent in the District of Columbia to a high of 94.7 percent in South Dakota. 
Four states, Hawaii, Maryland, New York and Rhode Island, and the District of Columbia had 
response rates below 80.0 percent. For Round 2 the rates ranged from a low of 69.3 percent 
in the District of Columbia to a high of 96.8 percent in Wyoming. No states had response 
rates below 80 percent. 

Table 5.4 shows Round 1 weighted teacher response rates for private schools by association 
group. The lowest rate observed, 58.1 percent, was for teachers in schools included in the 
area sample. For the list sample, response rates ranged from 61.2 percent to 86.6 percent. 
Round 2 response rates by association group are not available. Weighted response rates for 
affiliation groups in Round 2 were 87.9 percent for Catholic schools, 80,3 percent for other 
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religious schools and 78.1 percent for non-sectarian schools (Gruber, Rohr and Fondelier, 
1993, Table VI-4). 

The base for each of the teacher response rates presented in Tables 5.2 to 5.4 was the number 
of sample teachers who turned out to be eligible for the Teacher Survey. It excludes all 
teachers in schools that did not provide lists for sampling and it excludes school staff who 
were sampled but did not turn out to meet the survey definition of teacher or were no longer 
teaching at the sample school at the time the questionnaires were distributed. Table 5.5 shows 
the percentages of the initial sample that were excluded for such reasons. 

Table 5.6 shows unweighted item response rates for public and private school teachers in 
Rounds 1 and 2, Because of changes in questionnaire content, the rates for the two rounds 
are not exactly comparable. In Round 2, the proportion of items with response rates of 90 
percent or better declined somewhat, but the lowest observed item response rates were higher 
than they had been in Round 1. The patterns of item nonresponse were fairly similar for 
public and private school teachers. 

In Round 1, two questionnaire items had response rates lower than 75 percent, for both public 
and private schools. The first of these was an item asking for second major or minor field of 
study for each degree reported. Teachers responding to the questionnaire were asked to enter 
a code 00 if they had no second major or minor field. Apparently, many of them simply left 
the item blank. 

The second item with low response was the space for entering the total of a set of items 
asking for teachers to report their hours spent in school, during the most recent full week, on 
5 categories of school-related activities. Because of data reporting problems, the data for this 
entire set of items, which also covered time spent on school-related activities after school 
hours, were excluded from the public-use data tapes for the Teacher Survey. 

Most of the low item response rates observed on the Round 2 Teacher Survey questionnaires 
were for items that asked teachers who had answered "yes" to a question to report a related 
number or amount. For example, in an item on teacher training, several teachers who 
reported that they had taken courses in one or more of the subjects listed failed to enter the 
number of such undergraduate and graduate courses that they had taken (or to check the box 
for "none 11 in one of these categories). Similarly, teachers who reported that they had 
received certain types of income frequently failed to report the amounts. 

Measurement error: findings from reinterviews The first reinterviewing for the Teacher 
Survey occurred in conjunction with a large-scale pretest for Round 1 of SASS in the early 
part of 1987. The pretest, which covered 10 states, included 2,300 teachers in 220 public 
schools and 600 teachers in 75 private schools. A systematic sample of 127 teachers 
interviewed in the pretest was selected and telephone reinterviews were attempted by Census 
Bureau field representatives. Reinterviews were successfully conducted for 121 of them, for a 
95 percent completion rate. In the reinterviews the teachers were asked how they had 
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interpreted and answered selected questionnaire items, the extent to which they used or could 
have used records in responding, how much confidence they had in the accuracy of their 
responses and whether they had any recommendations for improving the questionnaire. 

The report of the reinterviews (Nash, n.d.) included several recommendations for 
improvements in specific questionnaire items: 

• In answering questions about college level courses taken, about half of the teachers 
reinterviewed said they had referred to records and a large proportion of the rest said 
they had records available. As a result of this finding, an instruction was added to the 
relevant items on the final questionnaire for Round 1, "Please refer to records if you 
cannot accurately recall your coursework." (For further information on the accuracy of 
responses to questions about degrees and courses taken, see "Measurement error, the 
Teacher Transcript Study 11 , later in this section.) 

• The pretest question on mathematics and science courses asked for number of credit 
hours completed. Most teachers felt it would be easier to report number of courses, 
and this recommendation was followed on the final questionnaire for Round 1 . 

For a question on how the teacher's classes were organized, comments in the 
reinterviews led to the addition of a new category, pull-out classes (where teachers 
provide instruction to students who are released from their regular classes), to the final 
version of this item for Round 1 . 

For an item on time spent in school-related activities, the re interview showed that 
about 2 *n f> teachers, in responding, had not included time spent away from school on 
such activities. As a result, separate sub-items covering time for certain kinds of 
activities away from school were added to the final version. 

(Note: For the following discussion of re interview results, readers not familiar with the 
interpretation of statistical measures of response variance developed from reinterviews may 
wish to refer to the side bar explaining these measures, in Chapter 2, p. 2.14.) 

As stated earlier, telephone reinterviews were conducted, covering selected items from the 
questionnaire, for slightly over one percent of the sample school teachers interviewed in each 
round of SASS. There were many differences between Rounds 1 and 2 in the topics covered 
in the Teacher Survey reinterviews. In particular, the number of opinion items covered was 
much smaller in Round 2 than it had been in Round 1. Four topical areas were covered in 
both rounds - educational attainment, full and part-time teaching experience by sector (public 
and private), current teaching assignment and plans to continue teaching. Even for these 
areas, however, there were some significant changes in question wording or format in Round 
2, 
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Table 5.7 shows reinterview findings for both rounds for the questions on teachers' 
education! attainment. The format for these questions was substantially revised between 
Rounds 1 and 2. As was the case for the School Administrator Survey (see Chapter 3, 
Section 3.3 and Table 3.6), the data suggest that the revisions led to more reliable and 
accurate reporting on bachelor's and master's degrees in Round 2, even though deficiencies in 
Round 1 for teachers had been less serious than for school administrators. The same could 
not be said for the reporting of associate degrees and educational specialist or professional 
diplomas; in fact, the index of inconsistency for associate degrees was somewhat higher in 
Round 2. 

Table 5.8 shows Round 1 and Round 2 reinterview results for questions on years of teaching 
experience, full and part-time, in the public and private sectors. In both rounds, the two full- 
time questions had relatively low response variance, but this was not true for the questions on 
part-time teaching. Despite the use of a redesigned format for the part-time questions in 
Round 2, no significant improvements were noted and the gross difference rate for part-time 
teaching in private schools actually increased somewhat. 

It is difficult to compare the reinterview results for Rounds 1 and 2 for the questions on 
teaching assignment, since they were substantially changed. For plans to remain in teaching, 
the same qi -.stion was used in both rounds and the estimates of the gross difference rate and 
the index oi inconsistency were both significantly higher for Round 2. The gross difference 
rate rose from 39.5 percent to 46.8 percent and the index of inconsistency rose from 55.4 to 
66.6. 

Table 5.9 shows the distribution of estimated indexes for all items covered in the Teacher 
Survey reinterviews in each round. In Round 1, about two-thirds of the items included were 
opinion questions and, as shown in the tab'e, nearly all of them had indexes in the high range 
(values of 50 and over). These items cov^ ;d teachers' views about topics like problems in 
their schools, their influence on school and classroom policies and practices, and the extent to 
which school administrators and other teachers had been helpful to them. In Round 2, only 
three opinion items were covered in the Teacher Survey reinterviews. As noted in Chapter 3, 
Section 3.3, it was felt that reinterview results for factual questions would be of more value 
for identifying problem questions and guiding efforts to improve their wording and format. 

One of the factual items included in the reinterviews in Round 2 asked teachers to report the 
grade levels for their current classes. There were 16 possible response categories, with an 
instruction to mark each one that applied. For the purpose of estimating indexes of 
inconsistency, each of the 16 categories had to be treated as a separate item. All of the 13 
categories for which estimates could be made had indexes in the low range, which was not 
surprising for such a relatively straightforward item. The data in Table 5.9 for factual items 
in Round 2 are shown with and without this item. When it is excluded, the distributions for 
factual items in Rounds 1 and 2 are somewhat similar. 
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Aside from the opinion items and the topics covered in reinterviews for both rounds, which 
we have already discussed, the topics with high response variability, as measured by 
reinterviews, were pay incentives in Round I and non-teaching income, courses and 
certification in Round 2. Further information about the accuracy of self-reported information 
on courses and certification is available from a record-check study, which is described in the 
next subsection. 

M easurement error: the Teacher Transcript Study In Round 2 of SASS, an experiment was 
undertaken to compare the accuracy of teachers' self-reports about their educational 
backgrounds with data obtained from transcripts of their college records (Chaney, 1993a,b). 
The data items to be compared for the two methods included degrees awarded, year of award, 
major and minor fields of study and number of courses taken or credit hours earned in four 
separate areas: education, area of main teaching assignment, area of second teaching 
assignment, and science and mathematics. 

The study was carried out "off-line' 1 , that is, a separate sample of teachers was used for the 
experiment. Two versions of the Teacher Survey questionnaire were administered, one asking 
for information on number of courses taken and one asking for information on number of 
credit hours earned in the relevant fields. Out of the initial sample of 867 teachers, 32 were 
later found to be ineligible for the study. Of the 835 eligible teachers, 592 (71 percent) 
agreed to participate in the study and provided names of the colleges they had att' ded, so 
that transcripts could be requested from these colleges. 

The teacher questionnaires were administered by the Census Bureau, using the standard 
mailing and telephone followup procedures. The request for permission to obtain 
respondents' transcripts came at the end of questionnaire. Thus for telephone interviews, the 
knowledge that this was to be done was unlikely to have influenced responses about degrees 
and courses. It is possible that some of the mail respondents could have gone back and 
checked their responses to these items after they discovered that their transcripts would be 
obtained. 

A total of 1,835 transcripts was requested. A transcript was provided for 74 percent of these 
cases and for 3 percent the college said it had no record of the identified person having 
attended. (For about one-third of the latter group, the teacher's attendance at the college 
could be confirmed on the basis of transfer notations on another college's transcript.) in 4 
percent of the cases the college said it could not locate the records and for the remaining 19 
percent the college did not respond to the request. The colleges also provided 168 transcripts 
that were not requested. These were generally instances where the same person had both 
undergraduate and graduate work at the institution, but did not report both on his or her 
questionnaire. 

For the 592 sample teachers who participated in the study, all requested transcripts were 
obtained for 51 percent, some but not all transcripts were obtained for 41 percent and no 
transcript information was obtained for the remaining 8 percent. For some of the data items, 
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such as degrees awarded, partial transcript information was sufficient to confirm self-reported 
data. However, if a teacher's self-reported degree was not confirmed, it would be difficult to 
conclude that the self-report was incorrect unless all requested transcripts for that teacher had 
been received. 

The data from the comparisons indicated that self-reports of types and years of degrees earned 
and major fields were, for the most part, accurate. However, information on numbers of 
courses and credit hours was less accurate. The study report says: 

Other errors appeared to show bias on the part of the respondent. For example, 
though there were errors in both directions, the general pattern was for teachers to 
overstate their preparation in their second teaching assignment and in mathematics and 
science as compared with the records on their transcripts. Since courses were coded as 
falling within the specified areas if there were any ambiguity, this overstatement is the 
rever-.s of what might be expected if there were simply differences between the 
teachers and coders in how to classify courses (Chaney, 1993a, p.20). 

Any proposal to rely on transcripts as the primary source of information on courses for 
sample teachers would, of course, have to take into account the additional costs associated 
with the collection of transcript data and the likelihood of higher item nonresponse resulting 
from failure of teachers to report all of the colleges where they had taken courses and failure 
of some colleges to supply the requested transcripts. 

Measurement error: cognitive research In the spring of 1990, Census Bureau staff members, 
using an early version of the 1990 pretest questionnaire, conducted "think-aloud" interviews 
with twenty teachers, ten from public schools and ten from private schools, representing 
various grade levels and specialties (Bates and DeMaio, 1990). The teachers were asked to 
verbalize their thoughts while they completed the self-administered questionnaires. The 
Census staff members asked questions as needed to understand how the teachers were 
interpreting the questions and what they were including in their answers. Findings from these 
interviews and from the subsequent pretest significantly influenced the development of the 
final questionnaires for the Round 2 Teacher Survey. 

Common occurrences noted by the Census interviewers were that teachers often failed to 
follow skip instructions (telling them to pass over questions that did not apply to them) and 
that they frequently failed to check boxes for "none", either leaving the item blank or entering 
"0" in the space reserved for an amount or number. For the most part, these errors were 
unlikely to bias the survey estimates, because the correct responses could be inferred, during 
data processing, from other entries on the questionnaire. However, attempting to answer 
questions that did not apply was observed to be frustrating to responding teachers and might 
in some instances lead to a decline in the perseverance of their efforts to pmvide complete 
and correct answers as they proceeded through the questionnaire. 

One possible solution to the problem of the failure to skip inapplicable questions is to use 
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redundant instructions, for example, using both a skip instruction next to the answer spaces in 
the item where the skip begins (the branching item) and an instruction at the beginning of 
each item that only applies to respondents with certain characteristics. The pretest 
questionnaire had some redundancy of this kind; the Census Bureau staff interviewers 
recommended additional uses of redundant instructions. 

Numerous problems were observed with an item for teachers who were teaching subject 
matter (departmentalized) courses to different groups of students. This item asked responding 
teachers to enter, in a matrix format, several items of information for each separate class they 
had taught in the most recent full week of teaching. The instructions for the items to be 
reported for each class appeared on the page preceding the matrix for recording the items. 

The word "class" itself caused difficulty because some teachers interpreted it to mean class 
period rather than, as was intended, a group of students receiving instruction in a subject 
during one or more class periods in the reference week. One of the specific items requested 
for each class was the number of graduation units associated with it. The instructions for this 
item asked the teacher to enter a code (0 for no credit, 1 for less than one unit, 2 for one unit, 
etc.), but some teachers entered the actual number of units rather than the code, an error not 
likely to have been corrected in data processing. 

Difficulties were also observed for an item about hours spent on school-related activities 
during and after school hours. As noted earlier in this chapter, a similar item in Round 1 had 
numerous response problems and the results were not included in the public-use data tapes. 
The main kinds of problems noted were: 

Answering in terms of hours per day, rather than for a reference week. 

Errors due to misunderstanding of the question format, which called for reporting time 
spent during and after school hours separately and, in each case, providing a total and 
a breakdown into two or more categories. 

This content of this item was substantially reduced and modified in the final questionnaire for 
Round 2, 

The Census Bureau staff report on these interviews included several recommendations for 
changes in specific items and for additional research on some of the kinds of response 
problems that were observed. 

Measurement error: findings from other sources Additional indications of measurement error, 
some systematic and some anecdotal, were provided by reviews of unedited and edited 
questionnaires and reviews of outputs from processing operations (clerical edit, computer pre- 
edit, computer edit and imputation). Such findings generally supported and extended what 
was learned in more formal evaluations through reinterviews, cognith e interviews and record 
checks. 
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Over 900 Teacher Survey questionnaires from the 1990 SASS field test were reviewed to 
identify items that were misunderstood by respondents or were difficult for them to answer 
(Fondelier and Bynum, 1990). The reviewers found several indications that respondents were 
concerned about the length of the questionnaire: notes to this effect on the forms, partially 
completed questionnaires and information on reasons for refusals. They also observed that the 
quality of data for mail responses appeared to be much better than that of the responses 
obtained by telephone followup. This was attributed in part to the unsuitability of the 
questionnaire design and format for telephone interviews, especially when the interviews had 
to be completed with teachers at their places of work, and in part to failure of the telephone 
interviewers to follow skip instructions and to complete items correctly and legibly. 

The specific item found by the reviewers to have the most problems was the one asking for 
information about classes taught in departmentalized courses (see also the comments on this 
item in the preceding discussion of cognitive interviews). Several respondents misinterpreted 
the meaning of "class 1 ' and several clearly failed to read the instructions that appeared on the 
page preceding the one on which the answers were to be recorded. For some elementary 
music, art and physical education teachers, insufficient lines were provided to record the data 
requested for each of their classes. 

The reviewers made numerous specific proposals for changes in the wording and format of 
questions and instructions. Many of their recommendations, as well as some of those 
emanating from the cognitive interviews, were followed in the final questionnaires for Round 

2> 

Information on both measurement and item nonresponse error is available from a 1992 review 
of post-edit item response rates, pre-edit reject rates and edit change tallies from Round 2 of 
SASS (Jenkins, 1992a). This review, which covered all four of the basic surveys, showed 
that, in comparison with the other surveys, pre-edit reject rates for the Teacher Survey were 
relatively low, but that item nonresponse rates were somewhat higher than for the other 
surveys. Nonresponse was relatively high for several parts of the item on classes in 
departmentalized courses. This item had already been identified, in cognitive interviews and 
questionnaire reviews prior to data processing, as having significant response problems. The 
edit change tallies showed that, as had been ob c ^n/ed at earlier stages, respondents frequently 
answered items that did not apply to them. 

5.4 Data processing and estimation 

Data processing procedures The sequence and nature of the data processing operations for the 
Round 2 Teacher Survey were similar to those described for the other three basic surveys in 
Chapters 2 to 4 (see Exhibit 2.1 in Chapter 2). As part of the clerical edit, codes were 
assigned to occupation and industry entries for teachers whose prior job had been outside the 
field of education. One difference for the Teacher Survey was that data entry keying was 
verified for a sample of one-third of the questionnaires, v '.^reas 100 percent verification was 
used for the other surveys. 

5.15 



ERiC 



In some instances, the clerks who reviewed the questionnaires that were rejected in the 
computer pre-edit operation were required to contact the teachers in an attempt to resolve 
discrepancies and omissions for critical items. The most frequent reasons for pre-edit 
rejection of questionnaires were inconsistencies in responses relating to full-time and part-time 
employment status and failure to respond to a question on main teaching assignment. 

Following the main computer edit and prior to imputation, an interview status edit was 
performed. Questionnaires were classified as out-of-scope if, for any reason, they were not 
members of the target population for the Teacher Survey (see Section 5.2, above). 
Questionnaires for in-scope teachers were classified as non-interview and excluded from the 
tabulations if one or more of the following conditions was not met: 

• The teacher reported the year that he or she started working as an elementary or 
secondary teacher. 

• At least one part of the educational background section had an acceptable response. 

• The teacher reported his or her main assignment field and whether or not he or she 
was certified in that field. 

The teacher reported at least one grade level of students currently being taught by him 
or her. 

• There were responses for at least 30 percent of the minimum items that a teacher 
should complete. 

Teachers whose questionnaires did not meet these minimum requirements were treated as 
nonrespondents in computing the unit response rates presented in Section 5.3, above. 

Imputation Round 2 imputation procedures for the Teacher Survey followed the same general 
pattern as imputation for the School Survey (Chapter 2, Section 2.4). Some missing or 
inconsistent values were imputed during the computer edit and, in a ^few cases where the 
correct entry was obvious, items were changed without contacting respondents in the initial 
clerical edit and the resolution of rejected cases from the computer pre-edit. However, most 
of the imputation was done in a computer operation following the computer edit. Computer 
imputation proceeded in two stages. In the first stage, missing or previously blanked values 
for selected items were imputed by using other information for the same teacher or making 
assumptions about the respondent's intended answer, for example, that not answering a 
question implied a response of "no". In the second stage, a hot deck procedure was used to 
impute the remaining missing values. The matching variables used to form imputation groups 
for each item and the order of their collapsing (when necessary to form sufficiently large 
imputation groups) arc described in Part VIII of the Round 2 Data File User's Manual, 
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In Round 1 of the Teacher Survey, there was no imputation of missing or blanked values 
following the computer edit and no flags were assigned to items imputed during the computer 
edit or earlier stages of processing. In Round 2, flags were assigned to all items imputed in 
the computer imputation operation. Items imputed in the first phase were flagged with code 
1, for "internal imputation" and those imputed in the second phase were flagged with code 2, 
for "donor-based" imputation. Items imputed prior to the computer imputation were not 
flagged. 

Weighting The overall weights for teachers in Round 2 were the product of six components. 
The basic sampling weight was the inverse of the teacher's overall selection probability, that 
is, the product of the school selection probability and the probability of selecting the sample 
teacher within the school. The sample adjustment factor accounted for unusual circumstances, 
such as mergers, splits or duplications, that had affected the school's probability of selection. 

Two components were designed to minimize nonresponse bias: a school nonresponse 
adjustment factor to account for schools that did not provide teacher lists for sampling and a 
teacher nonresponse adjustment factor to account for sample teachers for whom acceptable 
questionnaires were not obtained. The frame ratio adjustment factor was designed to reduce 
sampling error by adjusting sample estimates based on frame counts of teachers in sample 
schools to agree with the corresponding frame counts based on data for all schools. Finally, a 
teacher adjustment factor was used to force agreement between estimates of total number of 
teachers based on the School and Teacher Survey questionnaires. Unlike the other five 
components, this adjustment factor had not been used for teacher weights in Round 1 of 
SASS. 

Each of the last four factors was computed and applied within weighting cells comprised of 
schools or teachers with similar characteristics. Detailed descriptions of the weighting cells 
and the rules for collapsing them when necessary are provided by Kaufman and Huang 
(1993). 

In a review of the teacher weights for Round 2, some of the CCD teacher counts used in the 
numerator of the frame ratio adjustment factor were found to be one-tenth of the correct 
values, possibly as a result of data keying errors. The problem was worst in Iowa and the 
teacher weights for that state were recomputed. The problem may have existed in other 
states, but, given the difficulty of identifying these cases and the late stage at which the 
problem was discovered, no other corrections were made (Kaufman and Huang, 1993). 

The teacher adjustment factors used in Round 2 showed substantial variation by adjustment 
cell: 

Type of school Maximum Minimum Median 

Public 1.498 0.711 1.074 

Private 1.478 0.850 1.153 
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The maximum and minimum values for these adjustment factors were constrained by 
collapsing rules, which required that any cell with a factor outside the range from 0.667 to 
1.500 be collapsed with another cell according to prescribed rules. For the majority of cells, 
the teacher adjustment factors were greater than 1.000, indicating that teacher counts reported 
in the School Survey exceeded the number of teachers included on the teacher listing forms 
for the same schools. (For information about recent research on the completeness of teacher 
listings, see the discussion of "Coverage improvement" in Chapter 7, Section 7.3.) 

Variance estimation A balanced half-sample replication procedure (see Chapter 2, Section 2.4 
for details) is used to estimate sampling errors for all SASS surveys. Replicate weights for 
use in such estimates of sampling error are included on all SASS public-use and other 
microdata files. For sample teachers in schools not selected with certainty, the teacher 
replicates are the same as the school replicates. Sample teachers in certainty schools were 
split into two half-samples for the purpose of forming replicates for use in variance 
estimation. 

5.5 Evaluation of estimates 

This section describes comparisons of Teacher Survey estimates with data from other sources, 
including the School Survey, the Common Core of Data (CCD) and, to a limited extent, data 
available from other organizations. Some of these comparisons were made as part of pre- 
publication reviews; others were made subsequent to publication. 

Evaluation of estimates: Round 1 Prior to publication, estimates of full-time equivalent 
(PTE) teachers by state based on the Teacher Survey were compared with estimates from the 
School Survey. For most states, the Teacher Survey estimates were lower. One reason for 
this was that the number of teachers listed on the form used to sample teachers was often less 
than the count of teachers reported for the same school in the School Survey. In addition, 
there was evidence that many schools provided counts of FTE teachers that were too high. In 
the average state, 19 percent of the schools with some part-time teachers reported the same 
counts for total and FTE teachers (Choy, Medrich, Henke and Bobbitt, 1992). The extent of 
this phenomenon varied by state, from 10 percent in Alaska and Hawaii to 31 percent in 
Colorado (Kaufman, 1990). Consideration was given to the possibility of adjusting the 
Teacher Survey estimates to force agreement with CCD counts, but such an adjustment would 
not have resolved the discrepancies between the FTE teacher counts from the School and 
Teacher Surveys. 

The Round 1 Teacher Survey estimate of average hours spent in a week on school and 
school -related activities proved to be substantially below the corresponding estimate from the 
1985 Public School Survey, 40.3 hours versus 50.4 hours. There were several differences 
between the two surveys in the questionnaire items used to produce these two estimates. In 
addition, the Round 1 Teacher Survey estimates may have been low because there was no 
imputation when responses were provided for some but not all of the relevant items. 
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On the basis of this comparison, it was decided that no data on this topic would be published 
or included in the public-use microdata files (Hammer, 1990). 

Teacher Survey estimates of salary were compared with data available from private 
organizations. The Teacher Survey estimate of average base salary for public school teachers 
was $26,231, somewhat below the average salaries ($28,071) reported by the American 
Federation of Teachers, Swvey and Analysis of Salary Trends, I989 y (F. Howard Nelson, 
1990) and the National Education Association, Rankings of (he States, 1989. The higher 
figures from the latter 2 sources are believed to result from the inclusion, by some states, of 
other kinds of instructional expenditures in the category that covers teachers' salaries (Choy, 
Medrich, Henke and Bobbitt, 1992; Fowler, 1990). 

Evaluation of estimates: Round 2 The use of a teacher adjustment factor as one component 
of the Teacher Survey weights for Round 2 guaranteed that teacher estimates from the 
Teacher and School Surveys would agree for each of the weighting cells. However, as 
described in more detail in Chapter 2, Section 2.5, comparisons of School Survey public 
school FTE teacher estimates with counts from the CCD showed that estimates for nine states 
were at least 15 percent higher than the CCD counts. Investigation of this problem led to a 
series of additional processing and reweighting steps to make SASS state estimates of the 
number of public schools more cons ^nt with the CCD counts and to make enrollment and 
teacher count data consistent with the CCD on a school-by-school basis. These steps included 
reclassifying, as out of scope, teacher file records for teachers who taught only grades that 
were no longer considered part of a sample school and reinstating records for teachers at 
sample schools previously classified as out-of-scope because of apparent mergers (Fondelier, 
1992). 

Final School Survey estimates, by state, of full-time equivalent teachers were compared with 
counts from the 1990-91 CCD. At the national level, the SASS estimates exceeded the CCD 
count by 2.8 percent. The SASS estimates for South Dakota and Wisconsin were 31.7 
percent and 17,1 percent, respectively, above the CCD counts for those states. For 9 states, 
differences were in the range from 10 to 15 percent, with SASS being higher in all 9 states. 
For all other states, differences were less than 10 percent. The SASS and CCD data by state 
are shown in Chapter XII of the Round 2 Data File User's Manual (Gruber, Rohr and 
Fondelier, 1993). 
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Table 5.1 Teacher Survey Overall Response Rates 



Sector 


School 
Response 
Rate 17 


Teacher 
Response 
Rate^ 


Overall 
Response 
Rate* 




m w * mm 




Wfeww-v^::' . :<-..-:•::•::: 

>■ • 


Puhlir 




K6 4 


83 


Private 


88 


79.1 


70 






• 

• 


" 5£:V-:£:£> 

....... . 


Public 


95 


90.3 


86 


Private 


90 


83.6 


75 



Notes: 

1. Percent of ail in-scope schools providing teacher lists for sampling, unweighted. 

2. Percent of eligible sample teachers responding, weighted. 

3. Product of school and teacher response rates. 



Sources: 

NCKS (1991c). 

Gruber, Rohr and Fondelier (1993). 
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Table 5*2 Teacher Survey Response Rates- 





Round 1 (1988) 


Round 2 (1991) 


Unweighted 


Weighted 


Unweighted 


Weighted 


Public 


86.5 


86.4 


91.5 


90.3 


Private 


77.0 


79.1 


83.1 


83.6 



Notes: 

1. Percent of eligible teachers in sample who responded. 



Sources: 

Round 1 Unweighted: Kindel (1989). 

Round 1 Weighted: NCES (1991c). 

Round 2: Gruber, Rohr, and Fondelicr (1993). 
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Table 5.3 Teacher Survey Weighted Response Rates for Public Schools by State 





Response Rate 




Response Rate 


State 


Round 1 


Round 2 


State 


Round 1 


Round 2 


Alabama 


90.1 


90.6 


Montana 


90.6 


95.0 


Alaska 


90.1 


89.8 


Nebraska 


93.1 


92.9 


Arizona 


90.v 


94.9 


Nevada 


91.1 


88.5 


Arkansas 


9(U 


94.1 


New Hampshire 


85.2 


92.5 


California 


83.8 


87.9 


New Jersey 


80.9 


86.3 


Colorado 


88.9 


95.2 


New Mexico 


84.9 


90.0 


Connecticut 


80.2 


85.6 


New York 


74.6 


79J 


Delaware 


86.2 


95.6 


North Carolina 


88.6 


96.0 


District of Columbia 


68 >6 


69.3 


North Dakota 


93.3 


95.8 


Florida 


87.3 


88.7 


Ohio 


87.7 


87.8 


Georgia 


87.1 


93.3 


Oklahoma 


89.8 


93.8 


Hawaii 


74.1 


88.3 


Oregon 


94.3 


91.3 


Idaho 


92.6 


95.2 


Pennsylvania 


88.2 


93.3 


Illinois 


90.7 


95.6 


Rhode Island 


75.4 


87.4 


Indiana 


92.3 


95.3 


South Carolina 


89.0 


91.1 


Iowa 


93.0 


96.2 


South Dakota 


94.7 


95.2 


Kansas 


90.6 


95.6 


Tennessee 


85.1 


92.9 


Kentucky 


86.0 


88.8 


Texas 


87.3 


91.5 


Louisiana 


81.3 


93.1 


Utah 


89.9 


97.9 


Maine 


91.8 


89.7 


Vermont 


86.9 


95.6 


Maryland 


74.2 


90.2 


Virginia 


87.0 


90.7 


Massachusetts 


84.7 


84.4 


i Washington 


89.9 


88.1 


Michigan 


86.4 


84.5 


West Virginia 


87.7 


94.8 


Minnesota 


89.8 


94.1 


Wisconsin 


88.6 


95.3 


Mississippi 


88J 


93.3 


Wyoming 


91.2 


96.8 


Missouri 


87.9 


91.2 


TOTAL 


. 86.4 


90.3 



Source: NCES (1991c) and Gruber, Rohr and Fondelier (1993). 
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Table 5.4 Teacher Survey Weighted Response Rates for Private Schools by Association 
Group: Round 1 



Association Group 


Response Rate 
(Percent) 


Total 


79.1 


Area Sample 


58.1 


Association of Military Colleges and Schools - US 


85.4 


Catholic 


84.3 


Friends 


84.6 


Episcopal 


81.7 


Jewish 


63.8 


LfiiiQcran 


86.7 


Q»**t_'tmtli Hqv AHvPntictc 

ot> cii iii-uuy /\u vciiuaia 


81.4 


^Ki*icfiQn QpHnnlc fnf pruiafinri q1 

V^III IMlitll kJdIUUla til lCiii<t llUii<tl 


86.6 


American Association of Christian Schools 


61.2 


National Association of Private Schools for Exceptional 
Children 


71.3 


American Montessori Society 


79.4 


National Association of Independent Schools 


82.9 


Other 


74.3 



Source: NCES (1991c). 
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Tab'e 5.6 Teacher Survey Unweighted Item Response Rates 



Sector 


Range of Item 
Response Rates 
(Percent) 


Percent of Items with 
Response Rates: 


> 90% 


<75% 


Roond 1 .; ' '■ '". J ^'0 T : ,' 


Public 


64 - 100 


90 


1 


Private 


60 - 100 


89 


1 


Roood 2 


Public 


76 - 100 


84 


0 


Private 


71 - 100 


79 


1 



Sources: 

NCES (1991c). 

Gruber, Rohr, and Fondelier (1993). 
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Table 5.8 Teacher Survey Extent of Consistency Between Survey Interview and 
Reinterview, on the Topic "Years of Teaching" 





Gross Difference 


Index of 


Topic 


Rate 


Inconsistency 


(Summary of questions is 


Round 1 


Round 2 


Round 1 


Round 2 


presented beiowf 


( 1 

^ i yoo ) 


( 1QQ11 

V 1 77 1 f 




(1991) 


Fuli-time, Public 












7.6 


7.0 


10.8 


9.8 


90% Confidence 










Interval 


6.1*9.5 


8.7-13.4 


O *T t 1 4 

8.7«13.4 


1 1 I 1 4 


Part-time, Public 










Pnint Kvtimate 




6.6 


44.4 


42.5 


90% Confidence 










Interval 


6.M2.0 


5.0-8.6 


33.2-59J 


32.5-55.7 


Full-time, Private 










Point Estimate 


5.2 


5.3 


12.4 


8.8 


90% Confidence 










Interval 


3.6-7.4 


3J-8.7 


8.7-17.7 


5.4-14.4 


Part-time, Private 










Point Estimate 


3.4* 


7.5* 


38.5 


37.8 


90% Confidence 










Interval 


2.1-5.8 


4.8-11.6 


23.0-64,4 


24.4-58.4 



*Statislically significant difference between Round 1 and Round 2 (at 90% confidence). 

Source: Bushery, Roycc, and Kasprzyk (1992). 

Round 1 question for years of teaching: 

Including the current school year, how many years have you been employed as a teacher in public 
and/or private schools at the elementary or secondary level? 



Schools 


Years of full-time 
teaching 


Years of part-time 
teaching 


Public 






Private 







For round 2, four individual questions were used: 

♦ years teaching full-time in public schools 

♦ years teaching part-time in public schools 

♦ years teaching full-time in private schools 

♦ years teaching part-time in private schools 

For this analysis» responses were grouped into four categories, as follows: 

♦ less than three years 

♦ three to nine years 

♦ 10 to 20 years 

♦ more than 20 years 
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Table 5.9 Teacher Survey, Indexes of Inconsistency- Estimated from Reinterviews 



Round and 
Type of Item 


Number of 
Items 


Index of Inconsistency 


High 
>50 


Medium 
20-50 


Lo*v 
<20 


NA- 


'Round! ";' ■■. \T- ' :: : ' .".'C/-- 


Factual 


20 


5 


4 


3 


8 


Opinion 


42 


39 


3 






Round 2*' ' 'Jtj : 


Factual, ait 


53 


10 


14 


21 


8 


Factual, excl. 
item 29 


37 


10 


14 


8 


5 


Opinion 


3 


1 


2 







Notes: 

1. Each item cither had closed multiple-response categories or was converted to the equivalent by 
assigning class intervals to open-end responses. For items with more than 2 response categories, 
the L-foId index of inconsistency was estimated. 

2. Did not meet the minimum requirements to compute a reliable estimate of the index or 
inconsistency. 

3. Questionnaire item 29 asked about grade levels for the responding teacher's current classes, with 
16 possible response categories. Since the teacher was asked to mark each category that applied, 
item 29 had to be treated as 16 separate items for the purpose of estimating indexes of 
inconsistency. 

Sources: Newbrough (1989), Royce (1992). 
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CHAPTER 6 



THE TEACHER FOLLOWUP SURVEY 



6.1 Introduction 

The SASS Teacher Followup Survey is conducted in the sc? 30I year following the four basic 
surveys. Information is collected from a subset of the sample teachers who responded to the 
Teacher Survey in the base school year. On the basis of inquiries to their schools early in the 
following school year, teachers who responded in the base year are classified into three 
categories: 

(1) Leavers. Those who left the teaching profession between the base year and the 
following year. 

(2) Movers. Those who moved to a different school between the base year and the 
following year. 

(3) Stayers, Those teaching in the same school in both years. 

For some purposes the stayers and movers are referred to collectively as current teachers and 
the leavers as former teachers. The sample for the Teacher Followup Survey consists of all of 
the leavers and a subset of the movers and stayers. 

The main purposes of the Teacher Followup Survey are: to measure attrition rates for 
elementary and secondary teachers; to determine and compare the characteristics and attitudes 
of leavers, movers and stayers; to determine the current economic activities of leavers; and 
to obtain data on educational activities and future plans for all groups. Two different mail 
questionnaires were used for the survey in both rounds, one for current teachers (stayers and 
movers) and one for former teachers (leavers). The questionnaire for current teachers 
included a set of items that applied only to movers. A single version of the quest ionnai re was 
used for telephone followups of nonrespondents. 

The Teacher Followup Survey questionnaires for Rounds 1 and 2 included a request for 
information that would facilitate future contacts with the sample teachers, but there has been 
no further collection of information from them. There will be no recontacts of respondents to 
the Teacher Followup Survey following Round 3, but it is hoped that subsequent rounds may 
include some recontacts of respondents. 

For the four basic surveys (covered in Chapters 2 to 5), we have described the survey designs 
and procedures for Round 2 of SASS, along with information on the quality of data for both 
Rounds 1 and 2. For most features of the Teacher Followup Survey, however, we will 
describe only the design and procedures used in Round 1. The Round 2 Teacher Followup 
Survey was conducted for school year 1991-92 and data processing has been completed. A 
description of its design and procedures, along with some information on response rates and 
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other aspects of quality, is included in the Data File User's Manual for the Round 2 Teacher 
Followup Turvey (Whitener, Rohr, Bynum, Kaufman and King, 1994). 

The remaining sections of this chapter cover: frame development and sampling (6.2); data 
collection procedures and associated errors (6.3); and data processing and estimation (6.4). A 
section on evaluation of survey estimates is not included because no information on 
comparison of weighted survey estimates with data from other sources is available at this 
time. 

6.2 Frame development and sampling 

The target population The target population for the Teacher Followup Survey consisted of 
persons who, during the base school year (1987-88 for Round 1), were regular full-time and 
part-time teachers whose primary assignment was teaching in kindergarten or any of grades 1 
to 12, in eligible schools. Also included were persons who, in the base year, were substitutes 
filling the role of a regular teacher on a long-term basis or itinerant teachers (those teaching 
regularly in more than one school). 

The target population is divided into three groups: stayers, movers and leavers, according to 
their status in the year following the base year. Stayers are those who continued as teachers, 
according to the above definition, in the same school Movers are those who continued as 
teachers in a different eligible school. Leavers include all base-year teachers who were not 
teaching in kindergarten or grades 1 through 12 in the following year, including those who 
continued to work in schools, but in non-teaching jobs. 

Design considerations A primary sample design objective for the Teacher Followup Survey 
was to support comparative analyses of stayers, movers and leavers for teachers classified by 
sector (public and private), level (elementary and secondary), and years in teaching (new and 
experienced). A large majority of teachers in all categories were stayers (estimated at 86.6 
percent of public school teachers and 77.7 percent of private school teachers in school year 
1987-88) (Bobbin and Burns, 1991, Table 4). Consequently, it was necessary to ovcrsample 
movers and leavers in order .o reduce the sampling errors of estimated differences among 
groups. As mentioned in Chapter 5, Section 5.2, new teachers in private schools had been 
oversampled for the Teacher Survey in order to ensure a sufficient sample of teachers in this 
category for the Teacher Followup Survey, 

The Teacher Followup Survey does not include a sample of teachers who did not respond in 
the Teacher Survey. Base-year information would not be available for these teachers, thus 
limiting the utility of their 1 eacher Followup Survey responses for analysis. 

Frame development for Round 1 Before selecting a sample of teachers for the Teacher 
Followup Survey, it was necessary to determine their current status as stayers, movers or 
leavers. In late October 1988, the Census Bureau mailed computer-generated Teacher Status 
Forms to the school principals or heads of 1 1,584 schools nationally, requesting this 
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information for all sample teachers who had responded to the Teacher Survey, Schools not 
responding were telephoned to obtain the information requested for the teachers listed on the 
forms. For all teachers reported as having moved, the Census Bureau attempted to obtain 
their current home addresses from the U.S. Postal Service. 

Sample design and selection for Round 1 The sample of responding teachers for the Teacher 
Survey was the starting point for selecting the sample for the Teacher Followup Survey. 
Details about the selection of the sample of schools for the School Survey and the sample of 
teachers from those schools for the Teacher Survey can be found in Chapters 2 and 5, 
respectively. 

The samples of teachers for the Teacher Followup Survey were set at approximately 5,100 for 
public schools and 2,100 for private schools. These totals were further allocated within each 
sector among 12 strata defined in terms of current status (stayer, mover or leaver), level 
(elementary or secondary) and years of experience (new or experienced). A primary goal of 
the allocation was to have a sufficient sample of teachers in each of the 24 categories to 
permit comparisons across strata, for example, proportions of leavers among new elementary 
school teachers in public and private schools. 

Once the information on the current status of teachers who participated in the Teacher Survey 
was determined from their schools, the teachers were allocated to the 24 strata. All teachers 
whose current status had not been determined were classified as leavers for sample selection 
purposes. The sampling intervals needed to achieve the target sample sizes in each of the 24 
strata were calculated. For all of the leaver strata and some of the mover strata it was 
necessary to include all Teacher Survey respondents in the sample. 

In each of the strata for which a subsample of the Teacher Survey respondents was to be 
selecteu, the responding sample teachers were sorted in a specified order: for public schools 
by Census region, urbanicity, subject taught and school enrollment; and for private schools by 
association, urbanicity, subject taught and school enrollment. The samples for the Teacher 
Followup Survey were selected systematically, with probability proportionate to size. The 
measure of size used was the inverse of the teacher's probability of selection for the Teacher 
Survey sample, so that the Teacher Followup Survey samples for each of the strata would be 
more nearly self-weighting, that is, each teacher in a stratum would have the same base 
weight, prior to adjustments for nonresponse. 

Evaluation of the sampling frame The overall coverage of the target population for the 
Teacher Followup Survey depended in large part on the completeness of coverage of the 
frames used for the base-year School and Teacher Surveys. Evaluation of those frames is 
discussed in Chapters 2 and 5, respectively. The proportion of the target population for the 
Teacher Followup Survey covered by its sampling frame was further reduced by the exclusion 
of nonrespondents to the Teacher Survey. 

The current status of some teachers, as determined from their survey responses, may have 
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been different from the status reported for them on the Teacher Status Forms that were sent to 
the schools at the start of the 1988-89 school year. Such differences could result from 
changes in status during the school year or from reporting errors on the Teacher Status Forms 
or the Teacher Followup Survey questionnaires. Differences of the first two kinds would not 
bias the estimates, but would lead to increases in sampling errors as a result of the 
introduction of unequal sampling probabilities within some of the 24 strata used for sampling. 
The problem would be particularly severe if leavers or movers had been incorrectly classified 
as stayers, in which event they would receive base weights substantially greater than those of 
other teachers in their categories. Incorrect reporting of status on the Teacher Followup 
Survey questionnaire would, of course, bias the survey estimates. 

Assigning all teachers whose current status was unknown to the leaver strata also caused some 
increase in sampling error to the extent that such teachers turr *d out to be stayers or movers. 
However, the increase would have been much larger if these teachers had been assigned to the 
strata for stayers or movers. 

6,3 Data collection procedures and associated errors 

Data collection procedures for Round 1 Teacher Followup Survey questionnaires were mailed 
to the samples of current and former teachers at their home addresses in March 1989. For 
teachers not responding to the first mailing, a second set of questionnaires was mailed about 4 
to 5 weeks later. In the initial mailing, teachers who had been sent questionnaires that were 
inappropriate for their status (current or former teacher) had been asked to return them so that 
the correct version could be sent to them. These replacement questionnaires were sent at the 
time of the second mailing. 

Lists of nonrespondents to the mail questionnaires were sent to the Census Bureau regional 
offices for telephone followup by Census Bureau field representatives, starting in May 1989. 
For the telephone followups a separate version of the questionnaire, designed to accommodate 
both current and former teachers, was used. At this time the field representatives also tried to 
contact teachers for whom questionnaires had not been mailed because no current mailing 
address had been obtained. Means of locating such teachers included calls to the contact 
persons listed by the teachers on their Teacher Survey questionnaires, use of telephone 
directory assistance, and calls to the schools where the teachers had been teaching in the base 
year. All followup efforts were closed out at the end of the first week of July 1989 (FaupeK 
Bobbin and Friedrichs, 1992). 

Quality assurance In Round 1, reinterviews were attempted for 1,500 teachers, about 1 in 5 
of those who responded to the Teacher Followup Survey. They were successfully completed 
for 83 percent of the eligible cases. For teachers who responded by mail prior to the cutoff 
date, the reinterviews were conducted by telephone from the Census Bureau's Hagerstown, 
Maryland Telephone Center. For all other 'sachers, Census Bureau field representatives 
conducted the reinterviews by telephone. Results of the reinterviews are presented below, 
under the heading "Measurement error: findings from reinterviews." 
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Nonresponse error Table 6. 1 shows response rates, by sector and teacher status, for the 
Round 1 and Round 2 Teacher Followup Surveys. The overall response rates shown in the 
table are the product of response rates at three stages: obtaining teacher lists from schools, 
obtaining response in the Teacher Survey from a sample of the teachers listed, and obtaining 
response in the Teacher Followup Survey from a sample of those who responded in the 
Teacher Survey. Overall response rates improved between Rounds 1 and 2 in both the public 
and private sectors and for both current and former teachers in each sector. 

As the table shows, responses were obtained in both rounds from well over 90 percent of the 
teachers selected, in all categories, for the Teacher Followup Survey. Because of difficulties 
in locating former teachers (leavers), their response rates were lower than those for current 
teachers (movers and stayers) (Kaufman, 1991). Response rates at this stage were about the 
same for public and private school teachers. Because of lower response rates for pn*"*te 
schools and teachers in the first two stages, private school teachers' overall responr £s 
were substantially lower in both rounds than those for public school teachers. 

Table 6.2 shows unweighted item response rates, for current and former teachers, for Rounds 
1 and 2 of the Teacher Followup Survey. The rates for the two groups are not directly 
comparable, because there were substantial differences in the content of the two versions of 
the questionnaire. In Round 1, one low-response item was common to both versions: it 
asked whether there were any persons, other than spouse and children, dependent on the 
responding teacher for more than half of their financial support. The response rate for this 
item was 49 percent for both current and former teachers. For current teachers, only 65 
percen' responded to an item that applied only to movers who had moved to a private school, 
asking for the religious affiliation of that school. All other items on both versions of the 
questionnaire had response rates of 70 percent or more. 

Item response rates for a series of items asking for the level of respondents' satisfaction with 
various aspects of their current jobs were substantially higher for movers and stayers 
combined (99.0 to 99.4 percent) than they were for leavers (83,0 to 90.6 percent) (Choy, 
Medrich, Henke and Bobbin, 1992, p. 154). Some leavers, of course, did not have jobs for 
which these items would have been relevant and the questionnaire had a skip instruction 
designed to allow them to bypass this item. The lower item response rates for leavers may 
have been associated with some confusion about whether to skip and which set of items to 
skip. 

For Round 2, there were 3 items on the questionnaire for current teachers that had response 
rates less than 80 percent. All of them related to earnings from nonteaching jobs. For former 
teachers, there was only 1 item, asking for the kind of business or industry where the 
respondent worked, with a response rate below 80 percent. 

Measurement error: findings from re interviews {Note: For the following discussion of 
re interview results, readers not familiar with the interpretation of statistical measures of 
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response variance developed from reinterviews may wish to refer to the side bar explaining 
these measures, in Chapter 2, p. 2 J 4.) 

When asked in reinterviews to report their status at the time they responded to the initial 
interview, 7 percent of the teachers re interviewed reported a different status than they had in 
the initial interview. Of the 83 teachers who reported a different status, 20 changed from 
current teacher in the initial interview to former teacher in the reinterview and 63 changed 
from former to current teacher. No attempt was made to reconcile these differences in the 
reinterview. Because different sets of questions were asked for current and former teachers, 
those who reported a different status in the reinterview were excluded from further analyses 
of the questionnaire items included in the reinterviews (Royce, 1990). 

Table 6.3 shows the distribution of estimated indexes of inconsistency for all items included 
in the Teacher Followup Survey reinterviews, separately for current and former teachers. 
Most of the reinterview items for former teachers dealt with the teachers' opinions, attitudes 
and expectations. For current teachers there was a more nearly equal division between factual 
and opinion items. 

Most of the factual items had indexes in the low or medium ranges. The two factual items 
for current teachers that had high indexes of inconsistency related to teacher certification in 
the fields of their primary and secondary teaching assignments. Special analyses of the 
components of income reported by current teachers showed, for those who reported non-zero 
amounts on both occasions, a correlation of 0.95 for reports of base salary. For other 
components the estimated correlations were much lower: 0.22 for non-teaching compensation 
and -0.39 for summer school salary. 

The majority of opinion items had indexes of inconsistency in the high range and none of 
them were in the low range. Former teachers were asked to rate their current occupations on 
several aspects of job satisfaction both in an absolute sense and relative to teaching. Table 
6.4 compares the indexes of inconsistency estimated for the absolute and relative ratings. 
Even though the indexes were in the medium to high range for all items, respondents were 
clearly more consistent in providing comparative ratings on a three-point scale than they were 
in providing absolute ratings on a four-point scale. 

For items on current teachers' satisfaction with their jobs and on former teachers' satisfaction 
with their current jobs, all of which used a four-point scale, indexes of inconsistency were re- 
estimated with the four response categories collapsed into two: satisfied and dissatisfied. The 
resulting indexes were lower in all instances and in many cases moved from the high to the 
moderate range. As a result of these findings, the data from these items have generally been 
presented in the collapsed form in publications. 
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6.4 Data processing and estimation 



Data processing for Round 1 Data processing procedures were similar to those used in the 
four basic surveys. The main steps were: clerical edit, data keying, computer pre-edit, 
review and correction of rejects from computer pre-edit, and computer edit. The computer 
edit included range checks, inter-item consistency checks and a blanking operation to 
eliminate items that respondents answered unnecessarily because they did not follow skip 
instructions correctly. There was no imputation of missing items for Round 1 of the Teacher 
Followup Survey. 

Weighting in Round 1 The overall weights for teachers in Round 1 of the Teacher Followup 
Survey were the product of three components. The Teacher Survey final weight was ihz 
weight assigned to the teacher in producing the estimates for that survey (for a full 
description, see Chapter 5, Section 5.4). The Teacher Followup Survey basic weight was the 
inverse of the teacher's probability of selection, given that he or she had been selected for the 
Teacher Survey. The nonresponse adjustment was used to adjust for eligible sample teachers 
for whom questionnaires were not obtained in the Teacher Followup Survey. Within each of 
the 24 strata used in selecting the sample for the Teacher Followup Survey, the nonresponse 
adjustments were calculated separately for each of 12 adjustment cells defined by sex, level of 
education (2 categories) and age (3 categories) (Waite, 1990). 

The weights provided in the public-use data tape from the Teacher Followup Survey were 
slightly different from those used to produce tabulations published in Characteristics of 
Stayers, Movers, and Leavers: Results from the Teacher Followup Survey, 1988-89 (Bobbin 
and Burns, 1991). The resulting changes in the estimates were very small relative to their 
standard errors; most of the published percentages would not be affected (Faupel, Bobbitt, 
and Friedrichs, 1992, pp. 17-18). 

Variance estimation for Round 1 A balanced half-sample replication variance estimation 
procedure (see Chapter 2, Section 2.4 for details) is used to estimate sampling errors for ail 
SASS surveys. Replicate weights for use in such estimates of sampling error are included on 
all SASS public-use microdata files. Because the sample for the Teacher Followup Survey 
was a probability subsample of the Teacher Sample Survey, the same set of replicates was 
used for both surveys. However, some adjustments were made in the replicates for the 
Teacher Followup Survey in order to equalize the sample sizes for stayers, movers and leavers 
within each variance stratum* 

Variance estimates may be slightly biased because nonresponse adjustments and ratio 
estimation factors were not recalculated for each replicate, and no allowances were made for 
finite correction factors. Estimates of variances for small subdomains of interest are 
themselves subject to large sampling errors, especially when there are no data for the 
subdomain in some of the replicates in a variance stratum. 
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Table 6,1 Teacher Foliowup Survey Overall Response Rates: Round 1 





Sector 


Component 


Public 


Private 




Current 
Teachers- 


Former 
Teachers 


Current 
Teachers 1 ' 


Former 
Teachers 


ROUND 1 






School Response Rate*' 


95 


90 


Teacher Survey 
Response Rate* 


90.3 


83.6 


Teacher Foliowup 
Survey Response Rate 4 ' 


97.4 


92.4 


96.2 


94.1 


OVERALL 
RESPONSE RATE- 


84 


79 


72 


71 


ROUND 2 






School Response Rate 27 


96 


88 


Teacher Survey 
Response Rate 2 ' 


86.4 


79.1 


Teacher Foliowup 
Survey Response Rate 4 ' 


97.5 


93.6 


96.6 


93.1 


OVERALL 
RESPONSE RATE 5 ' 


81 


78 


67 


65 



Notes: 



1. Includes stayers and movers, 

2. Percent of all in-scope schools providing teacher lists for sampling, unweighted, 

3. Percent of eligible sample teachers responding to Teacher Survey, weighted. 

4. Percent of eligible sample teachers responding to Teacher Foliowup Survey, weighted. 

5. Product of first three components. 

Sources: NCES (1991c), NCES (1992c), Gruber, Rohr, and Fondelier (1993), Whitener, S., Rohr, C, 
Bynum, L,, Kaufman, S. and King, K. (1993). 
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Table 6,2 Teacher Followup Survey Unweighted Item Response Rates 



Status 


Range of Item 
Response Rates 
(Percent) 


Percent of Items with 
Response Rates*. 


> 90% 


< 80% 


Kounu 






Current 
Teacher 


65 - 100 


90 


5 


Former 
Toucher 


27 - 100 


61 


1 








Current 
Teacher 


67 - 100 


95 


5 


Former 
Teacher 


57 - 100 


87 


1 



Source: NCES (1991c), Whitener, S., Rohr, C M Bynum, L M Kaufman, S. and King, K. (1993). 
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Tabic 6.3 Teacher Followup Survey Indexes of Inconsistency- Estimated from 
Reinterviews: Round 1 



Teacher Status 


Number of 


Index of Inconsistency 


and Type of Item 


Items 


High 
>50 


Medium 
20-50 


Low 
<20 


K.I A V 

NA- 


Cturcttt (movers mtimym) ±} 










Factual 


19 


2 


4 


3 


10 


Opinion 


13 


8 


1 




4 


Former (leavers) 












Factual 


2 




1 


1 




Opinion 


22 


13 


7 




2 



Notes: 

1. For items with more than 2 response categories, the L-fold index of inconsistency was estimated. 

2. Did not meet the minimum requirements to compute a reliable estimate of the index of 
inconsistency. 

Source: Royce (1990). 
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Table 6.4 Teacher Followup Survey Indexes of Inconsistency for Selected Opinion 
Items for Leavers: Round 1 



Aspect of Current 
Occupation Rated 

/T* v f Slf Stl lACf tittle tc 

1 1 exi oi questions i» 
presented below) 


Index of Inconsistency When: 


Rated for 
Current 
Occupation-' 


Current Occupation 
Compared to 
Teaching^' 


Salary 

Point Estimate 
90% Confidence 
Interval 


63* 

54-74 


37* 
30-48 


Opportunities for 

Professional Advancement 
Point Estimate 
90% Confidence 
Interval 


63 

54-75 


56 

47-70 


Autonomy or Control 
Over Your Own Work 
Point Estimate 
90% Confidence 
Interval 


79* 

69-92 


53* 
43-65 


Benefits 

Point Estimate 

Ann/ £^ J ^ 

90% Contidence 
Interval 


65* 
56-76 


38* 

31-48 


Intellectual Challenge 
Point Estimate 
90% Confidence 
Interval 


60* 

51-72 


43* 

35-53 



•Statistically significant difference between absolute and comparative ratings (at 90% confidence). 



Notes: 

1, Question 27 . How satisfied are you with EACH of the foiiowing aspects of your CURRENT job? 
Are you (a) Very satisfied, (b) Somewhat satisfied, (c) Somewhat dissatisfied, or (d) Very 
dissatisfied with- 

2. Question 26 , How would you rate teaching relative to your current PRIMARY occupation in 
terms of EACH of the following aspects? Please indicate (a) Better in teaching, (b) Better in 
current position, or (c) No difference- 
Source: Royce (1990). 
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CHAPTER 7 



SUMMARY 



7.1 Introduction 

This report reflects our current knowledge of the quality of SASS data. Direct, quantitative 
measures are available for some components of error, including sampling error, simple 
response variance (from reintcrviews) and, for a few topics, reporting bias (from the Teacher 
Transcript Study). Only indirect measures are available for some other components of error. 
Unit and item nonresponse rates identify specific subpopulations and data items that are most 
likely to be affected by nonresponse bias. Comparisons among SASS and other surveys 
provide indications of the extent to which survey estimates may have failed to include some 
members of the target populations of school districts, schools, school administrators and 
teachers. As described later in this chapter, experimental and evaluation studies now 
underway will provide additional information. 

It is not feasible to combine all of these indications of quality in some way to develop precise 
point estimates of total survey error for individual data items. Nevertheless, we believe that 
the information on quality that is now available or is being developed will help users decide 
how much confidence to place in the estimates that are of interest to them and determine how 
best to use these data in their analyses. The Quality Profile has been developed primarily as a 
convenient source of information about quality for users of SASS data. 

We believe that the Quality Profile, with periodic updates, will also serve as a useful guide to 
the survey designers and managers at NCES and the Census Bureau in the effective allocation 
of additional resources to their continuing efforts to improve the quality of SASS data. There 
were many significant changes in the design, procedures and instrumentation used for the 
SASS surveys between the first and second rounds, and additional changes have been 
introduced in the' third round, covering school year 1993-94. Many of these changes are 
designed to improve the quality of SASS data, as disclosed by the direct and indirect 
measures and indicators of quality presented in this report. 

Section 7.2 of this chapter summarizes the information about different sources and 
components of error that has been presented in Chapters 2 through 5 for the four basic 
surveys and Chapter 6 for the Teacher Followup Survey. Section 7.3 describes additional 
research and evaluation activities that were underway at the time this report was being 
prepared. Section 7.4 presents some suggestions for data users on how to make effective use 
of SASS data products. Special attention is given to the possible effects of procedural and 
design changes on analyses of change between Rounds 1 and 2. 

7.2 Principal sources of error 

Covcrane error The units of analysis for SASS are schools, school administrators, public 
school districts (LKAs) and teachers. Coverage errors can occur when units in the survey 
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target populations are omitted from sampling frames, when they appear more than once m the 
frames (if the duplication is not discovered prior to release of the estimates) or, in the case of 
schools and school districts, when a single unit in the target population is treated as more than 
one unit in the sampling frame, or vice versa. Coverage errors for schools can have a direct 
effect on the coverage of teachers and school administrators and, for public schools, may also 
affect the coverage of LEAs. 

The target population for the School Survey was redefined between Rounds 1 and 2, in 
conjunction with the change in the frame used for selection of the public school sample. The 
Quality Education Data (QED) list used in Round 1 defined schools primarily in terms of 
physical location, whereas the Common Core of Data (CCD) list used in Round 2 defines 
them in terms of administrative units, so that it is possible to have more than one school at a 
single physical location or a single school with more than one location. 

There are no direct estimates of gross or net coverage errors available for any of the SASS 
surveys. Comparisons of SASS estimates with data from other NCES surveys provide some 
indications of possible coverage error. As described in Section 2.5, SASS Round 2 estimates 
of the number of public schools by state were compared with the CCD counts. For the total 
U.S., the SASS estimate was 97.9 percent of the CCD count for the same school year. For 
most states the SASS estiir .es were within 5 percent of the CCD counts. Other things being 
equal, one would expect the CCD counts of public schools to be somewhat higher than the 
SASS estimate for the same year because the list frame for the School Survey was based on 
the CCD for the second year preceding the reference year, and therefore did not include all 
schools that started operation in the reference year or the preceding year. However, other 
factors, such as inclusion of out of scope units in the CCD may have operated to cause 
differences in the other direction. Similar considerations apply to the list frame used for 
private schools. 

As described in the four chapters covering the base-year surveys, there are several indications, 
some of them quantitative, of potential coverage error. These include: 

The use, for both the public and private school surveys, of list frames constructed two 
years prior to the reference school year for the survey (Section 2.2). 

The need to use an area sample to supplement the list frame for private schools. The 
area sample accounted for about 22 percent of the estimated number of private schools 
in Round 1 and about 21 percent in Round 2, indicating no significant improvement of 
coverage by the list frame in Round 2 (Section 2.2). 

In Round 2, it was discovered that some multi-site special education programs of the 
State of California had been listed on the CCD as single schools. Adjustments were 
necessary to eliminate duplication for those sites located at existing schools and to 
select a sample of the other sites (Section 2.2). 
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Discovery in both rounds, subsequent to sample selection, of some duplicate listings in 
the private school list frame. (Section 2.2). 

• In Round 1, exclusion from the QED frame of 275 small Nebraska LEAs with about 
2,800 students (Section 2.5). 

For the teacher surveys, use of teacher listing forms that ask only for teachers working 
at the sample schools at the time the forms were being completed. Teachers who 
begin working later in the reference year have no chance of inclusion (Section 5.2). 

In both rounds, counts of teachers on the teacher listing forms were, on the average, 
lower than the counts reported for the same schools on their School Survey 
questionnaires (Sections 5.2 and 5.4). 

Sample estimates of the number of schools were also affected in both rounds by school survey 
respondents who provided data for a unit other than the one intended on the basis of the 
sample selection. Some respondents reported combined data for two different schools at the 
same location, and some, especially in small LEAs, reported combined data for all schools in 
the LEA, Conversely, in the Teacher Demand and Shortage Survey, a few LEAs reported 
data for a single school rather than the entire LEA. Many of these erroneous reports were 
identified and corrected prior to data release, but some may have escaped detection (see 
Sections 2,5, "Prepublication checks" and 4.3, "Measurement errors associated with data 
collection'*). 

Nonresponsc error Unit and item response rates for each of the five surveys have been 
presented in Chapters 2 to 6, To permit comparisons among the surveys, Table 7,1 presents 
response rates for all five surveys by round, separately for the public and private school 
sectors. Response rates for public schools have consistently exceeded those for private 
schools. Response rates in Round 2 exceeded those for Round 1 for each of the four basic 
surveys and the Teacher Followup Survey, for both sectors. As mentioned in Chapter 2, these 
increases may have resulted in part from the more lenient criteria used in Round 2 in 
accepting questionnaires that had missing or incomplete responses for some items. Response 
rates for the Teacher and Teacher Foliowup Surveys are composite rates, reflecting losses 
from schools that did not supply teacher lists and nonresponding teachers from schools that 
did supply lists. Consequently these rates were, with one exception, lower than those for the 
other three surveys. 

As shown in additional tables in Chapters 2 to 6, there was considerable variation in response 
rates within each sector. For the public school sector in Round 1, in each of the four basic 
surveys a few states had response rates of less than 80 percent. This was due in part to a 
small number of LEAs, some of them fairly large, that declined to participate in any of the 
surveys (Section 2.3). In Round 2, all states had response rates above 80 percent in the 
School Survey and the School Administrator Survey (see Tables 2,6, 3,2, 4,2 and 5,3), For 
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the private school sector, one or more association groups had response rates of less than 60 
percent in each of the four basic surveys in Round 1 (see Tables 2,7a, 3.3a, 4.3 and 5.4). 
Comparable data for the Teacher Survey are not available for Round 2. 

Adjustments for unit nonresponse are included in the estimation weights for all of the surveys; 
however, the success of such adjustments in reducing bias depends on the extent to which the 
characteristics of units that respond and do not respond are similar. Survey results for 
domains of analysis with low response rates should be interpreted with caution. A new 
analysis of the characteristics of nonrespondents and the possible effects of unit nonresponse 
will be released soon (Scheuren, Parke and Bureika, 1994). 

Data on item response rates for the five surveys were presented in Tables 2.9, 3.5, 4.5, 5.6 
and 6.2. These rates cannot be compared across surveys in a meaningful way because of 
differences in content. There appears to have been some reduction in item nonresponse 
between Rounds 1 and 2 for the School, School Administrator, Teacher Demand and Shortage 
(public school sector) and Teacher Followup Surveys. This improvement probably resulted in 
part from dropping items that proved especially difficult for respondents in Round 1. 
Changes in questionnaire format may also have contributed to the reduction in item 
nonresponse. For the Teacher Survey, in contrast, item nonresponse rates were somewhat 
higher in Round 2. With one exception, the items flagged as having low response rates in 
Round 2 were also used (some with slightly different formats) in Round 1 of the Teacher 
Survey, so there is no obvious explanation for the lower response rates observed for these 
items in Round 2. 

Examination of the questionnaire items with the lowest response rates in each survey and 
round suggests that factors associated with item nonresponse include question format, 
respondent burden and sensitivity. "None" boxes were frequently overlooked, as were spaces 
for entering amounts associated with positive responses to a yes or no question, especially 
when the amount spaces were located well to the right of the yes and no boxes. 

Responses were frequently incomplete for complex "matrix-style" items, an example being an 
item on staffing patterns in the Round 1 School Survey questionnaires. This item called for 
counts of teachers by subject and by their status in the reference and prior school years. Data 
for this item and a similar one in the Teacher Demand and Shortage Survey questionnaire 
were judged to be of such poor quality that they were not included on the public and 
restricted use data files for those surveys. Because of the high nonresponse and other 
reporting problems, questions on this topic were substantially simplified for Round 2. 

Items for which low response nvw have been associated with item sensitivity included several 
from the Round 2 Teacher Survey relating to amounts of teacher income from sources other 
than teaching. Another one was an item from the Round 1 Teacher Followup Survey that 
asked teachers who had moved to a nesv school for that school's religious affiliation, if any. 

In Round U most missing or inconsistent items were imputed for the School and the Teacher 
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Demand and Shortage Survey questionnaires, but there was only limited imputation for the 
other three surveys. In Round 2, missing items were imputed for all surveys. All items 
changed in the computer imputation phase of data processing (but not those changed in 
preceding operations) were flagged as being imputed on the public and restricted-use data 
tapes. Some missing items can be deduced with a fairly high degree of accuracy from other 
responses on the same questionnaire. Other items are imputed by the "hot-deck" method, 
which assumes the values of missing items to be similar to those reported by other units that 
have the same basic attributes. In Round 1, for the School and the Teacher Demand and 
Shortage Surveys, a single code was used to flag imputed items. In Round 2, separate codes 
were used to distinguish internal and donor-based (hot deck) imputations. 

Measurement error Information about measurement (response) errors associated with data 
collection comes from several sources: reinterviews, a record-check study, in-depth interviews 
using cognitive research techniques, methodological experiments, reviews of completed 
questionnaires and analyses of errors and inconsistencies detected during data processing. 
Information for each survey from these sources is presented in Chapters 2 through 6. Here 
we summarize the main findings. 



Reintervievvs have shown that the items asking for the opinions, perceptions and future 
expectations of teachers and school administrators are, almost without exception, 
subject to high response variability. Moderate reductions in variability can be achieved 
by combining responses to 4-point scales into two categories (Sections 3.3, 5.3 and 
6.3). 

Evidence from several sources suggests that the quality of information obtained by 
mail is superior to that obtained in telephone followups to nonrespondents. There are 
several possible reasons for this: questionnaires were not explicitly designed for use in 
telephone interviews; some of the questions can be answered more accurately by 
referring to records, which is harder to do in a telephone interview; persons who do 
not respond by mail are less likely to be motivated to provide accurate information; 
and there were indications that the training and supervision for the telephone followup 
interviews could have been improved. In the Round 2 School Survey, about one-third 
of the public school questionnaires and nearly one-half of the private school 
questionnaires were completed by telephone. 

An experiment, the State Data Project, was undertaken in connection with the Pretest 
for Round 2 of SASS to test the feasibility of obtaining data for the public sector 
Teacher Demand and Shortage Survey from state rather than local education agencies. 
A comparison of data collected from both sources for the same sample of LEAs 
showed a high frequency of substantial differences (more than 10 percent in either 
direction) for several variables. The experiment did not include any means of 
determining which of the two sources had provided more accurate data. Based on 
these findings, it was decided not to try to collect data from state agencies in Round 2 
(see Section 4.3, "The State Data Project"). 
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Some of the concepts adopted for SASS data collection appear to be unfamiliar to 
respondents and to cause them considerable difficulty in formulating appropriate 
responses, One such concept is that of full-time equivalent (FTE) teachers used in the 
School and the Teacher Demand and Shortage Surveys. A school that has part-time 
teachers should report numbers of FTE teachers that are lower than their teacher 
counts. Nevertheless, many such schools reported the same numbers for teacher counts 
and FTE teachers (see Section 2.5, "Evaluation of published estimates: Round 1"). 

• A record-check study, the Teacher Transcript Study, compared teachers' self-reports of 
their educational backgrounds with data from their college transcripts. The main 
conclusion was that self-reports of types and years of degrees earned and major fields 
of study were reasonably accurate, but that self-reported information on courses and 
credit hours in specific fields was less accurate (see Section 5,3, "Measurement error: 
the Teacher Transcript Study"). 

• For all surveys and in both rounds of SASS, it was common for respondents to ignore 
skip instructions and consequently to try to answer questions that did not apply to 
them, Such errors have little or no direct effect on the quality of data, because most 
inapplicable responses can readily be deleted in clerical and computer edits. However, 
it is possible that frustration induced by trying to respond to irrelevant items may lead 
to a falling off, as respondents proceed through a questionnaire, in their level of 
commitment to providing complete and accurate information. 

The foregoing and other findings relating to measurement error led to numerous changes in 
survey instruments and procedures between Rounds 1 and 2, and additional changes were 
made for Round 3. Some of the questions identified as being especially difficult have been 
eliminated, reduced in scope or modified. For a few items which were found to have 
especially severe nonresponse or other reporting problems, no estimates were published and 
individual responses were eliminated from the public-use data tapes. Included in this category 
were: item 28 on the Round 1 Teacher Questionnaire, which asked for information about 
hours spent on school-related activities; items 9 and 10 on the Round 1 Teacher Demand and 
Shortage Questionnaire, which asked for detailed information about full-time equivalent 
teaching positions, by specialty; and item 32 (35 for private schools) on the Round i School 
Survey Questionnaire, which asked for teachers by primary field of assignment in the current 
and preceding school year. 

Data processing and estimation error In contrast to the sources of error discussed up to this 
point, there is relatively little documentation of processing and estimation errors. There are 
three clerical operations, each with the potential for both resolution of errors from earlier 
processing stages and introduction of new errors: clerical review of incoming questionnaires 
(in the Teacher Survey, this includes the coding of some industry entries), data entry, and 
resolution of rejects from the computer pre-edit operation. The first and third of these 
operations include some telephone contacts with respondents to obtain missing items or 
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resolve inconsistencies. Data entry is verified for 33 percent of the Teacher Survey 
questionnaires and for 100 percent of the questionnaires in the other three base-year surveys. 

There have been no formal studies or experiments with alternative imputation procedures for 
the SASS surveys. Information relevant to the weighting procedures used in the School 
Survey was obtained in a study by Shen, Parmer and Tan (1992). After examining the 
correlates of nonresponse in that survey, they recommended some changes in the definition of 
the nonresponse adjustment cells for the private school sector and in the order of collapsing 
small cells for both public and private schools (see Section 2,4, Weighting), Most of the 
recommended changes were made in defining the nonresponse adjustment cells in Round 3, 

As noted in Section 5,4, a new component was introduced into the weights used for estimates 
from Round 2 of the Teacher Survey, The purpose of this component, called the teacher 
adjustment factor, was to force agreement between teacher counts from the School and the 
Teacher Surveys, The differences prior to adjustment apparently are due primarily to the fact 
that schools do not include, on the teacher lists used for sampling, all of the teachers they 
include in their counts in the School Survey. If the teachers not included on the lists differ in 
some respects from those who are included, use of the adjustment factor will not necessarily 
eliminate all biases resulting from their not being represented in the sample. 

Sampling error At present, there are two ways for users of SASS data to determine the 
sampling errors of estimates that are of interest to them. Users of data from publications will 
find that standard errors are provided for many of the published estimates. Users of 
microdata files can compute standard errors for any estimate by employing readily available 
software for variance estimation by the balanced half-sample replication method. Half-sample 
replication weights for this purpose are included in the microdata files. 

A recent study has confirmed the feasibility of including generalized variance functions in 
SASS publications (Salvucci and Holt, 1992; Salvucci, Galfond and Kaufman, 1993). These 
functions, which relate the sampling error of an estimate to its size, can be used by those who 
do not work with microdata files or lack the software for the replication method to produce 
approximations to the sampling errors associated with their estimates of interest. Specific 
parameter values for the four basic surveys in Round 1 have been computed and will be used 
for internal analyses. Parameter values for Round 2 are being developed and will be made 
available in a forthcoming NCES technical report. 

The balanced half-sample replication method assumes that sampling units have been selected 
with replacement, but in fact sampling without replacement is used in all of the SASS 
surveys. Violation of the assumption leads to overestimates of the true variances, but the 
effects are small unless the sampling fractions are quite large. Large sampling fractions do 
occur in the selection of samples of LEAs for the public sector Teacher Demand and Shortage 
Survey in some of the smaller states, so it is likely that sampling errors in those states will be 
substantially overestimated (see Section 4.4, Variance estimation). 
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Comparisons with data from external sources Comparisons of school, teacher and enrollment 
counts with other NCES surveys have been discussed in connection with "Coverage error" at 
the beginning of this section. Here we summarize comparisons of SASS data with 
information available from sources other than NCES: 

The Census Bureau collects data on school enrollment annually in the Octcbcr 
Supplement to the Current Population Survey (CPS), a monthly national sample survey 
of households. SASS estimates of private elementary and secondary school enrollment 
from Round 1 exceeded the CPS estimates for the same school year (1987-88) by 15 
percent. NCES surveys of private schools prior to SASS had shown similar 
differences with CPS enrollment estimates during the 1980s (see Section 2.5, 
Evaluation of published estimates: Round 1). 

The National Catholic Education Association conducts an annual census of Catholic 
schools. SASS Round 1 estimates of the number of Catholic schools and their 
enrollment exceeded the Association's census counts by 6.1 and 7.8 percent, 
respectively (see Section 2.5, Evaluation of published estimates: Round 1). 

Public school administrators' salaries reported in the Round 1 School Administrator 
Survey were compared with data obtained directly from state education agencies in 
selected states. The two sets of data were not fully comparable; however, the patterns 
were similar and there were no obvious inconsistencies (Section 3.5). 

Round 1 estimates of teachers' salaries were compared with data from private 
organizations. The Teacher Survey estimate of average base salary, $26,23 1 , was 
6.6 percent below a $28,071 estimate of average salary for the same school year from 
a 1989 survey conducted by the American Federation of Teachers and 6.4 percent 
below an estimated average salary of $28,029 reported by the National Education 
Association (see Section 5.5, Evaluation of estimates: Round 1). 

7.3 Current research 

This section describes several research, development and evaluation activities which are in 
various stages of completion. Some are ju' . ^tting underway. For others, data have been 
collected or compiled and the results are being analyzed. Included in this section is a 
description of a new reinterview procedure that was introduced in Round 2 of the Teacher 
Followup Survey. 

Expansion of coverage and content: library media centers and staff Two projects are related 
to plans to expand the coverage and content of SASS. As part of a pretest for Round 3 of 
SASS that was conducted during school year 1991-92, questionnaires for collecting data about 
public and private school library media centers and library media staff specialists were tested. 
The media center questionnaires included items on staffing, facilities, collections, equipment, 
expenditures, users and services provided. The questionnaires for library media specialists 
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asked for information about training and experience, current status and roles, perceptions and 
attitudes toward work, compensation, incentives and demographic characteristics. 

The pretest sample included 682 public and private schools. Questionnaires were returned for 
525 school media centers and 410 media specialists. The lower response for media specialists 
occurred in part because some schools do not have such persons on their staffs. Item 
nonresponse and other features of the pretest responses were analyzed and the questionnaires 
redesigned for use in Round 3 of SASS (Williams, 1992). 

Expansion of coverage and content: students Collection of data about students is another area 
of expansion for SASS. Data from school records for a sample of students are being 
collected from a subsample of schools in Round 3 (1993-94). The subsample for this 
component includes an oversample of Indian schools, public schools with high Indian 
enrollment, and public schools located in Alaska. The student data collected from school 
records will be linked to data for their schools and teachers. Topics covered include: 
students' demographic characteristics; disabilities; course work, including advanced placement 
courses; and participation in special programs (Colaciello, 1993b). 

Procedures for selecting samples of students and obtaining information about them from 
school records were tested in 1991. The initial sample for the study consisted of 200 public 
and private elementary and secondary schools. Of these, 192 were found to be eligible for 
participation and 174 (90.6 percent) of the eligible schools agreed to participate in the test. 

For each participating school, selection of a sample of students proceeded in three stages: 
selection of a sample of up to five teachers, selection of a day and class period for each of the 
sample teachers, and selection of a sample of five students from the rosters for the selected 
class periods. For the third stage, two different procedures were used. Half of the sclools 
were asked to submit a teacher's student roster for each of the selected class periods and the 
sample of students was selected by the Census Bureau. The schools were then asked to 
record information for the selected students on a student questionnaire. The other half of the 
schools were asked to select the student samples themselves, according to instructions that 
were included on the questionnaire to be used for recording the information about the sample 
students. 

For each student selected by either method, the schools were asked to provide information on 
demographic characteristics, current grade level, attendance, participation in special programs, 
disabilities and, for students in grades 9 to 12, grade-point average, type of program and 
current enrollment in mathematics and science courses. Information about the student's 
attendance at classes taught by each of the sample teachers in his or her school (sometimes 
called multiplicity information) was also requested to provide a basis for the determination of 
selection probabilities for the students included in the sample. 

Analysis of the sampling operations suggested that either procedure could be used 
successfully. Selection of the student sample by the Census Bureau maintains tighter control 
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over the process, but requires more time and an additional mailing to the schools (Frazicr, 
1992). 

A review of the data from the 1991 pilot test showed that the multiplicity items, which 
provide the data needed to weight the sample students correctly, were not producing high 
quality results. Research was undertaken, using cognitive interviews with school 
administrators and teachers, to develop and test improved versions of these items and to 
determine whether they could be more readily answered by school administrators or by 
teachers. Teachers were found to be better respondents for the multiplicity items. 

A field test of the new questionnaires and procedures was conducted in the spring of 1993, 
using a sample of 282 public schools and 194 private schools to test the collection of student 
data by mail with telephone followups. For a separate sample of 28 schools in 5 states, the 
sampling and collection of data for students were carried out by personal visits. The majority 
of this latter group were Indian Schools and public schools with high Indian enrollment 
(Colaetello, 1993a). 

Using the procedures developed in these tests, student data are being collected for a 
subsampie of schools in Round 3. Information about estimation procedures will be included 
in a forthcoming paper by King and Kaufman (1994). 

Periodicity Rounds 1 to 3 of SASS have been conducted at three-year intervals, but some 
thought has been given to the possibility of a different cycle. An initial exploration of the 
implications of cycles of varying leLjth was undertaken to provide guidance to the decision 
on when the second round of SASS should be conducted. Models were developed to explore 
the tradeoffs between the cost of a survey cycle and the errors of key estimates, with the cost 
depending on periodicity and sample sizes and the error being expressed as a composite of 
sampling error and the error of prediction based on prior year estimates. Several other 
factors, including response burden and the need for time to evaluate the Round 1 content and 
methodology, influenced the decision in favor of a three-year interval between Rounds 1 and 
2. 

The question of periodicity is now being reexamined. The assumptions underlying the models 
that were used previously arc being reviewed and the results of various optimization 
calculations based on estimates of key variables from Rounds 1 and 2 will be evaluated 
(Ghosh, Kaufman and Smith, 1994; Smith and Ghosh, 1994), 

Alternative reinterview procedure A new reintervicw procedure has been tested in Round 2 of 
the Teacher Followup Survey, For all previous reinterviews in Rounds 1 and 2, responses 
were obtained for selected questionnaire items, but no attempt was made to reconcile 
differences between responses given in the initial interviews and the reinterviews. The data 
from the original interviews and the reinterviews were used to estimate simple response 
variances and other measures of response consistency for the items included in the 
reinterviews, 
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The Round 2 reinterviews for the Teacher Followup Survey included a procedure for 
reconciling differences. Responses from the initial interviews were transcribed to the 
questionnaires used by the Census Bureau field representatives who conducted the 
reinterviews. After completing all of the selected items in the normal way, the field 
representatives were instructed to compare the interview and reinterview responses. For every 
item that had a different response, they were to try to determine, using a specified set of 
questions, which of the two responses was correct and why the difference occurred (Harris, 
1992a,b). This information will provide the basis for estimating both response variance and 
response bias, and is expected to be useful in improving the wording and format of the 
questionnaire items included in the reinterviews. Some initial results are given by Jenkins and 
Wetzel (1994a,b). 

Alternative modes of data collection Possible changes in the modes of data collection for 
SASS are being evaluated. As mentioned in Chapters 1 and 2, development and testing of 
computer-assisted methods of response for schools and LEAs has begun. Interactive diskettes 
with the survey questions will be mailed to respondents, who will complete Ihem using their 
own computers (see Section 1.2, Evolution of the SASS design). A prototype will be tested 
in a small sample of schools during Round 3 of SASS. This method of data collection has 
already been used successfully by the NCES for the completion, by state offices, of 
questionnaires relating to public libraries and the completion of questionnaires for academic 
libraries (Kindel, 1992). 

A first attempt to evaluate the feasibility of collecting data for LEAs from state education 
agencies was inconclusive. There were substantial differences between items reported directly 
by LEAs and the corresponding values reported by the state agencies. Further research would 
be needed to understand the reasons for the differences and to identify specific items which 
might be adequately reported at the state level (see Section 4.3, 'The State Data Project"). 

Improvement of response rates When telephone folio wups are necessary for teachers who do 
not mail in their questionnaires, it has proved difficult to reach them at their schools and 
complete the interviews by telephone while they are there. In the pretest for Round 3 of 
SASS, conducted during school year 1991-92, postcards were sent to teachers during the mail 
followup phase asking them to supply their home telephone numbers if they were willing to 
be contacted at home (Section 5.3, Test of new followup procedure). The proportion 
responding was low; hence this procedure is not being used in Round 3. 

A study is underway, using data from al! of the SASS surveys in Round 2, to compare the 
characteristics of nonrespondents and respondents, based on the sampling frame information 
that is available for both groups. It is hoped that the results of the study will suggest methods 
of improving response rates for problem groups and also possible improvements in the 
nonresponse adjustments used in developing estimates from the data for responding units 
(Moonesinghe, Smith and Gruber, 1993; Scheuren, Parke and Bureika, 1994). 
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Coverage improvement The quality of SASS data is affected in many ways by the quality of 
the sampling frames for schools, LEAs and teachers. Frame imperfections - onissions, 
duplications and incomplete or incorrect information about the characteristics of u.uts included 
in the frames - can cause both biases and increases in the sampling errors of the survey 
estimates. The target populations change over time and the frames must be updated to reflect 
these changes. 

Several current evaluation and research projects are aimed at the improvement of the sampling 
frames and other features of the SASS surveys that relate to coverage. For public schools and 
LEAs, the CCD has been adopted, starting in Round 2, as the frame of choice. As discussed 
in Section 2.2 (Evaluation of the sampling frames), some problems of omission, duplication 
and incomplete information on school and LEA characteristics have been encountered in using 
the CCD. A plan has been developed for a detailed assessment of the quality of data 
collected in the CCD surveys, including the data that are used to create and maintain the LEA 
and public school sampling frames (Nisselson, Parke, Streett, Salvucci and Fink, 1993; Peng, 
Gruber, Smith and Jabine, 1993). 

For private schools, NCES requested the Census Bureau to undertake a detailed analysis of 
private school list and area frames and the procedures for updating them (Bynum, 1992; 
Dillen and Jackson, 1992). Results of 1991 updating operations for both frames were 
evaluated to determine which sources and strategies were most effective for frame updates. A 
preliminary analysis of the additions to the list frame is available (Jackson, 1993) and 
additional results are presented by Jackson and Frazier (1994). 

In Rounds 1 and 2, there have been several instances of LEAs and public schools completing 
questionnaires for units other than those intended. For example, a school questionnaire may 
be completed for two different schools at the same physical location or for all of the schools 
in a small school district. An LEA may complete a questionnaire for a single school, rather 
than all of the schools under its jurisdiction. (For further detail, see Section 2.3, 
"Measurement error: findings from in-depth interviews", Section 2.5, and Section 4.3, 
"Measurement errors associated with data collection".) Work is continuing on efforts to 
redesign the instructions and initial items on the school and LEA questionnaires to make it 
easier for respondents to identify the units for which they are being asked to report. 

Finally, as mentioned in Chapter 5, the numbers of teachers listed by the schools for sampling 
purposes are, on average, smaller than the teacher counts reported on the School Survey 
questionnaires (Section 5.2, Frame evaluation). During the 1992-93 school year an extensive 
test, the Teacher Listing Validation Study, was undertaken to seek answers to 3 questions: 

(1) What kinds of problems do schools have in completing the teacher listing 
forms? 

(2) For public schools, would LEAs be able to provide teacher listings that arc 
more accurate than those prepared by the schools? 
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(3) What is the relative accuracy of teacher counts from the school questionnaires 
and the teacher listing forms? 

The first two questions were investigated for a sample of 300 private schools, 290 public 
schools and 254 LEAs (some LEAs had more than one sample school). The third question 
was investigated for a separate sample of 300 public schools and 290 private schools. All of 
the schools in these two samples were asked to complete teacher listing forms and the LEAs 
were asked to complete teacher listing forms for the sample school(s) in their districts. 
Various techniques involving personal and telephone interviews for a subsample of schools 
were then used to investigate the study questions. The field phase of the Ttacher Listing 
Validation Study has been completed and some results are now available (Royce, 1993, 1994), 

Evaluation of estimates For several variables, SASS obtains information from more than one 
survey. Estimates of the number of teachers, for example, can be obtained from the School, 
Teacher, and Teacher Demand and Shortage Surveys. When aggregate estimates for school 
districts, states and other domains are compared, the differences are sometimes larger than 
could be accounted for by sampling variability, A Cross-Questionnaire Estimates Comparison 
Study is being undertaken to systematically document comparable estimates that can be 
produced from more than one SASS survey, compare them at several levels of aggregation, 
and identify possible reasons for differences (Kasprzyk and Seheuren, 1994; Fink, 1994). 

Additional information Results of these ongoing research, development and evaluation 
activities will be documented in internal memoranda, contractor reports and, where 
appropriate, in NCES working papers, technical reports and papers presented at professional 
association meetings or in journals. Based on this SASS Quality Profile, NCES is reviewing 
past and ongoing research on the quality of SASS data, with a view toward identifying gaps 
in our knowledge and establishing priorities for future research activities. For a forthcoming 
document based on this review and for further information about the status of specific 
projects, write to: 

SASS Quality Profile 

555 New Jersey Avenue, N,W, 

Washington, D,C 20208-5651 



7.4 Suggestions for users 

User options The three basic means of user access to SASS data are publications, public-use 
data tapes and restricted-use data tapes. Data tapes contain individual records for schools, 
school administrators, public school districts or teachers, so that users may tabulate or analyze 
the records as required to meet their specific needs and, with some restrictions, link data 
across surveys. The public-use data tapes can be obtained through the lnter-university 
Consortium for Political and Social Research. Their content is limited in order to protect the 
confidentiality of individual respondents. Data for Round 1 are also available on CD-ROM 
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and data for Round 2 will be issued in this format also. For information, write to: 

Superintendent of Documents 
U.S. Government Printing Office 
P.O. Box 371954 
Pittsburgh, PA 15250-7954 

Restricted-use data tapes contain additional information, allowing users to analyze data for 
more detailed geographic areas and for the complete set of private school association groups. 
Researchers desiring access to restricted-use data tapes must apply to NCES for a site 
licensing agreement to use the tapes. Write to: 

Associate Commissioner 
Statistical Standards and Methodology Division 
NCES/OERI, U.S. Department of Education 
555 New Jersey Avenue, NW 
Washington, DC 20208 

For detailed current information on SASS publications and how to obtain them, call 
1 (800) 424-1616. For information about the purchase of data tapes write to: 

U.S. Department of Education 
OERJ/EIRD/Data Systems Branch 
555 New Jersey Avenue, NW 
Washington, DC 20208-5725 

(202) 219-1522 or 219-1847 

Learning about SASS Even the most casual user of SASS data can benefit by learning 
something about the objectives, content and design of the SASS surveys, and how they relate 
to each other. A good starting point is the current version of An Overview of SASS and the 
TFS. 

Purchasers of data tapes for a survey receive the data file user's manual for that survey. The 
Round 1 manual for each of the four base-year surveys (NCES, 1991a,b,c,d) contains 
information on the design and procedures for all four surveys, plus the data base 
documentation and copies of the questionnaires for the particular survey. For Round 2, there 
is a single manual, in three volumes, covering all four of the base year surveys (Gruber, Rohr 
and Fondelier, 1993). Volume I provides general information about survey content, design 
and methodology; Volumes II and III contain the detailed specifications for the Restricted-Use 
Version and the Public-Use Version of the datafile, respectively. For each round there is a 
separate manual for the Teacher Followup Survey (Faupel, Bobbin and Friedrichs, 1992; 
Whitener, Rohr, Bynum, Kaufman and King, 1994 - public-use and restricted-use versions). 



7.14 



158 



Users of data from SASS publications are urged to read the technical notes and appendices 
included in those publications. Detailed technical notes for Round 1 of SASS are included in 
NCES Publication 92-120, Schools and Staffing in the United States: A Statistical Profile and 
in the corresponding publication for Round 2, Schools and Staffing in the United States: A 
Statistical Profile, 1990-91, NCES Publication 93-146. The publications in the ED. Tabs 
series contain technical notes and some include copies of questionnaires for the surveys on 
which they are based. Detailed accounts of the sample design and estimation procedures for 
Rounds 1 and 2 of SASS are given in technical reports by Kaufman, 1988 Schools and 
Staffing Survey Design and Estimation, NCES 91-127, and by Kaufman and Huang, 1990-91 
Schools and Staffing Survey: Sample Design and Estimation, NCES 93-449. 

A SASS User Group, consisting of SASS data users in the Washington DC area, meets about 
once a year. NCES representatives announce plans for the next round of the survey and 
solicit user opinions about the availability of data and ease of use. Once every year NCES 
representatives meet with members of private school associations to share relevant information 
with them and solicit their views on various aspects of the survey. A SASS Review Board, 
consisting of distinguished researchers, meets with the NCES staff periodically to provide 
advice on technical questions. 

Using cross-sectional data Conscientious analysts and researchers will want to have a 
thorough understanding of the nature and limitations of the data they are using. We 
recommend, of course, that they read the parts of this report that are relevant to the data they 
are working with. Some additional suggestions are: 

Review the questionnaires (available in the data file user's manuals) and examine the 
wording and format of the specific data items that are of interest to them. 

• Take sampling errors into account. What is the confidence interval for an estimate of 
interest? Could observed differences have occurred by chance or are they statistically 
significant? As noted in Section 7.2, "Sampling error", SASS publications include 
sampling errors for key items, and users of data tapes may, if they wish, estimate 
sampling errors for their variables of interest. Generalized variance functions, which 
provide approximations of sampling errors for all estimates, based on their size, will 
be available soon for Round 2 estimates. 

• Consider the possible effects of nonrcsponsc error on the estimates of interest. For 
example, in making comparisons of public school data by state or private school data 
by association group, take note of the substantial variations in unit response rates by 
state and by association group. For specific items, note the item response rates. The 
data tapes include flags which identify all items that were changed in the computer 
imputation phase of data processing. For Round 1, a single flag was used to identify 
all items imputed at this stage in the School and Teacher Demand and Shortage 
Surveys; most missing items were not imputed and no flags were provided for those 
that were imputed in the other surveys. In Round 2 missing items were imputed for 
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all surveys. Items that were imputed following the computer edit stage of processing 
were assigned flags that distinguish imputations based on other information for the 
same unit (internal imputation) from those based on information for other units with 
similar characteristics (donor-based imputation). If desired, some or all imputed values 
may be omitted from analyses or reimputed by alternate methods. 

Analyzing changes over time In working with data from the Round 2 surveys, many users 
will want to look at the changes that have occurred in the three-year interval between Rounds 
1 and 2. We encourage SASS data users to do this, but at the same time we urge them to be 
aware of the many changes in the content, design and procedures for the two rounds of 
surveys and to consider how these changes may affect estimates of change. SASS is a 
complex, evolving system of surveys: comparability over time is highly de'irable for periodic 
surveys, but changes are necessary at this early stage in order improve the quality of the data 
in ways suggested by the early indicators of quality that have been presented and discussed in 
this Quality Profile. 

In the chapter for each of the 4 base surveys we have included information on changes 
between Rounds 1 and 2 in the content, design and procedures for each survey. Following 
are some key points: 

As described in more detail in the introductory sections of Chapters 2 through 5, there 
have been numerous changes in content, with deletion of some topics, addition of 
others, changes in the kinds of information collected for each topic and changes in the 
wording and format of individual items and sub-items. Users interested in estimates of 
change are advised to obtain copies of the questionnaires for both rounds and review 
the questions and response categories for the items that are of interest to them. 

As described in Chapter 2, a new, more rigorous procedure was used in Round 2 to 
develop a locale or "urbanicity" code to describe the type of community in which each 
sample school was located. This locale classification will differ in many instances 
from the code for the self-report of community type obtained in both Rounds 1 and 2 
on the School and School Administrator questionnaires (Gruber, Rohr and Fondelier, 
1993, p. 137; Johnson, 1993). 

In Round 1 a separate Teacher Demand and Shortage Survey questionnaire was sent to 
private schools in the sample. In Round 2, a single questionnaire containing the 
questions for both the School and the Teacher Demand and Shortage Surveys was sent 
to the sample of private schools. As shown in Table 7.1, the response rate to the 
combined questionnaire in Round 2 was substantially higher than the response rate to 
the separate Teacher Demand and Shortage Survey questionnaire used in Round 1 . 

The shift from the use of the QED list as a sampling frame for schools in Round 1 to 
the CCD list in Round 2 was accompanied by a change in the definition of a school. 
The definition of a school for Round 2 was the same as the CCD definition, and the 
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public-use and analysis tapes use this definition. However, to permit comparisons, 
some tabulations will be prepared using both the QED and CCD definitions (Holt and 
Scanlon, 1994; Saba, Zhang and Chang, 1994). 

• In Round 1, virtually all missing or inconsistent data items were imputed for schools 
and school districts, but there was only limited imputation of missing data for school 
administrators and teachers. In Round 2, missing or inconsisteut data items were 
imputed for al! surveys. 

♦ Items changed in the computer imputation phase of data processing are identified on 
the data tapes by imputation flags for schools and school districts in Round 1 , but not 
for school administrators and teachers. Imputation flags are provided for all of the 
surveys in Round 2. The flags used in Round 2 distinguish imputations based on other 
data for the same units (internal imputation) from those based on data for other similar 
units (donor-based imputation). 

In Round 2, a new weighting factor was applied to data from the Teacher Survey to 
force agreement between estimates of teacher counts based on the School and Teacher 
Survey questionnaires. This weighting factor was not used in Round 1, with the result 
that estimates of teacher counts from the Teacher Survey were, in general, below those 
from the School Survey. As a result, estimates of change in numbers of teachers 
between Rounds 1 and 2 based solely on the Teacher Survey will, in general, be 
overestimates. 

* It is not necessarily safe to assume that measurement bias affecting specific items will 
remain constant over time and therefore will have little or no effect on estimates of 
change. There are indications that the effects of some kinds of measurement bias may 
have been smaller in Round 2. Table 7.1 shows that unit response rates increased for 
all of the base-year surveys in Round 2: as a result, biases associated with unit 
nonresponse are likely to have decreased. The systematic reinterview program has 
provided indications that improved wording and format of specific items has led to 
smaller response variance and bias for some items, one example being provided by the 
reports of degrees earned by school administrators and teachers, as discussed in 
subsections of Sections 3.3 and 5.3, both covering "Measurement error: findings from 
reinterviews 11 . 

The estimation of sampling errors for estimates of change is somewhat more complicated than 
for point estimates As a first approximation, the variance (square of the standard error) for 
an estimate of change can be taken as the sum of the variances of the Round 1 and Round 2 
estimates from which it was derived. In many instances this formulation will overestimate the 
true variance. It assumes no correlation between the two estimates, whereas there will in fact 
be some correlation because of the deliberate introduction of overlap in the samples of schools 
and school districts for the two rounds. More precise values and procedures for calculating 
sampling errors of est mated changes will be provided as they are developed. 
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User feedback If you are a user of SASS data, we invite you to let us know about your 
experiences in using the data, any problems you may have encountered, and your suggestions 
for improving the quality of data from any or all of the surveys. We also invite your 
comments on this Quality Profile. Has it been useful to you, a*d what additional information 
should we include in future versions to make it more useful? Please write to: 

SASS Quality Profile 
555 New Jersey Avenue, N.W. 
Washington, D.C 20208-5651 
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Table 7.1 Response Rates by Survey and Sector: Rounds 1 and 2 
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Public 


78 


79 
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65 


71 
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Notes: 

1. For (he private sector, the School and Teacher Demand and Shortage Survey questionnaires were 
combined in Round 2. 

2. Combination of weighted and unweighted rates reflecting losses at all stages. 
Sources: 

See Tables 2.5, 3.1, 4.1, 5.1 and 6.1. 



7.19 



BEST COPY AVAILABLE 

er|c ,6; 



I 
I 



REFERENCES 



Bates, N., and DcMaio, T. (1990), "Report on Cognitive Research on the Public and Private 
School Teacher Questionnaires for the Schools and Staffing Survey (SASS-4A and 
SASS-4B)." Internal Census Bureau memorandum for distribution list, July 13. 

Blank, R.K. (1992), "Improving Reliability and Comparability of NCES Data on Teachers and 
Education Staff: Results of a CCSSO/NCES Workshop." Presented at 1992 Annual 
Meeting, of the American Statistical Association, Boston, August 1 1. Council of Chief 
State School Officers, Washington, DC. 

Bobbitt, S.. and Burns, S. (1991), "Characteristics of Stayers, Movers, and Leavers: Results 
from the Teacher Followup Survey, 1988-89." E.D. Tabs, NCES 91-128, June. U.S. 
Department of Education, Office of Educational Research and Improvement, 
Washington, DC: National Center for Education Statistics. 

Box, G. (1993), "Quality Improvement -- The New Industrial Revolution." International 
Statistical Review, 61, 3-20. 

Bureau of the Census (1990), "1990-1991 Schools and Staffing Survey Sample Telephone 
Script." Census Bureau training document, October 1. 

-(1991a), "1991 SASS Recommendations." Collection of memoranda 

from Census Bureau regional offices in response to 1991 SASS request for 
recommendations, July 2. 

(1991b), "Schools and Staffing Survey: Field Representative's 

Instruction Manual, SASS-20." 

— — ■ — (1992), "1992 SASS Pretest: Supervisor's Comments." Collection of 

memoranda from Census Bureau regional offices in response to request for 
comments, June 10. 

Bushery, J., Royco, D., and Kasprzyk, D. (1992), "The Schools and Staffing Survey: How 
Reinterview Measures Data Quality." Proceedings of the Section on Survey Research 
Methods, American Statistical Association, pp. 458-463. Alexandria VA: American 
Statistical Association. 

Bynum, L. (1992). "1991 PSS Universe Coverage Improvement Evaluation-Project 7173." 
Internal Census Bureau memorandum for the record. 



R-l 



164 



Chancy, B. (1993a), "TheAccuracy of Teacher's Self-Reports on their Postsecondary 
Education." Unpublished report prepared for the National Center for Education 
Statistics, Westat, Inc., May. 

(1993b), "The Accuracy of Teachers' Self-Reports on their Postsecondary 

Education." Proceedings of the Section on Survey Research Methods, American 
Statistical Association, p. 692. Alexandria, VA: American Statistical Association. 

Choy, S., Henke, R., Alt, M., Medrich, E., and Bobbin, S. (1993), Schools and Staffing in the 
United States: A Statistical Profile, 1990-9L NCES 93-146. Washington, DC: U.S. 
Government Printing Office. Stock no. 065-000-00581-6. $14. 

Choy, S., Medrich, E., Henke, R., and Bobbin, S. (1992), Schools and Staffing in the 
United States: A Statistical Profile, 1987-88. NCES 92-120. Washington, DC: U.S. 
Government Printing Office. Stock no. 065-000-00508-5. %\2. 

Colaciello, L. (1993a), "1993 Stu4ent Records Research Field Test." Demographic Surveys 

Division, Census Bureau, March 29. 

yxX \ i f 

(199&); ,,, 1994 Student Records Survey." Demographic Surveys Division, 

Census Bureau*, September 20. 

Council for American Private Education (1992), Private ^chools of the U.S., 1992 Edition. 
Shclton, CT; . Market Data Retrieval, Inc. 

Dillen, K. and Jackson, B. (1992), ,l l 991 PSS Universe Coverage Improvement Analysis 
Proposal," Internal CensUs Bureau Memorandum for the record, September 21. 

Elliott, E.J. (1991), "Types of Publications-- A Revision of the technical Content and 

Structure of Publications Directive".. Internal NCIjS Memorandum to staff, March 28. 

"' * I 

Faupel, E>, Bobbin, S., and Friedrichs; K. (1992)^" 1988-89 Teacher Followup Survey: Data 
File User's Manual. 11 NCES 92-058. U.S. Department of Education, Office of 
Educational Research and Improvement, Washington, Dp: National Center for 
Education Statistics. * t J h 

Ferrel!, J.M. (1992), "Documentation of the SASS-10 Teacher Postcard for the 1992 SASS 
Pretest." Internal Census Bureau memorandum for the record, February 14. 

Fink, S. (1994), "1990-91 Schootyand Staffing Survey (SASS) Cross Questionnaire 
Analysis," forthcoming report to NCES. 




Fondelier, S. (1989a), "Correction of Respondent Error in Reporting Enrollment for the 1987- 
88 Schools and Staffing Survey (SASS)." Internal Census Bureau memorandum for the 
record, November 6. 

(1989b), "Correction of FTE Teacher Counts on the 1987*88 SASS Public 

School File (SASS-3A)." Internal Census Bureau memorandum for the record, 
November 27. 

(1990), "Meeting with SMD and NCES to Discuss Data Problem on SASS-1A 

(District) File." Internal Census Bureau memorandum for the record, February 13. 

(1992), "Summary of 1991 SASS Problem Where Estimates of Public School 

Enrollment and Teacher Counts were Inconsistent with the Common Core of Data." 
Internal Census Bureau memorandum for the record, October 28. 

Fondelier, S., and Bynum, L. (1990), "1990 Field Ttsl for the Schools and Staffing Survey - 
Review of the Public and Private School Teacher Questionnaires (SASS-4A and 4B)." 
Internal Census Bureau memorandum for the record, July 30. 

Forsman, G M and Schreiner, I. (1991), "The Design and Analysis of Rcinterviews: An 

Overview," in Measurement Errors in Surveys, pp. 279-301. Biemer, P., Groves, R., 
Lyberg, L., Mathiowetz, N., and Sudman, S., eds. New York: John Wiley & Sons, Inc. 

Fowler, W. (1990), "Teacher Salaries: Comparing States After Adjusting for Teacher 

Experience and Education." Paper presented to the annual meeting of the American 
Educational Research Association, Boston, MA, April 17, available from National 
Center for Education Statistics. 

Frazier, R. (1992), "1991 SASS Research Study: Final Report." Internal Census Bureau 
memorandum for the record, October 20. 

Ghosh, D., Kaufman, S., and Smith, W. (1994), "Optimal Periodicity of a Survey: Sampling 
Error, Data Deterioration, and Cost." Paper presented at the annual meeting of the 
American Statistical Association, section of Survey Research Methods, Toronto, 
Canada, August. 

Groves, R. (1 989), Survey Errors and Survey Costs. New York: John Wiley & Sons, Inc. 

Grubcr, K. (1992a), "1990-91 Final ISR aid Noninterview Rates by Regional Office." Internal 
NCES memorandum for the record, January 28. 

(1992b), "Private School Survey 1989-90: Comparison with NCEA Data." Internal 

NCES memorandum for the record, July 8. 



R-3 



16G 



Gruber, K., Rohr, C, and Fondelier, S. (1993), 1990-91 Schools and Staffing Survey: Data 
File User's Manual Volume I: Survey Documentation. NCES 93-144. Office of 
Educational Research and Improvement. Washington, DC: National Center for 
Education Statistics. 

Hammer, C. (1989a), "Checking Salary Data on SASS. M Internal NCES draft memorandum, 
December 26. 

(1989b), "Checking Estimated Numbers of Public and Private School 

Administrators on the 1987-88 SASS against Other Sources." Internal NCES draft 
memorandum, December 26. 

(1990), "Comparison of Hours Spent for Teachers on the 1985 Public School 

Survey, the SASS Teacher Survey, and the SASS Administrator Survey." Internal 
NCES draft memorandum, February 20. 

(1992), "Results of examination of 1990-91 SASS public and private school 

administrator files (SASS 2A and 2B). ,f Internal NCES memorandum for the record, 
October 21. 

Hammer, C, and Gerald, E. (1991), "Aspects of Teacher Supply and Demand in Public 
School Districts and Private Schools: 1987-88." E.D. Tabs, NCES 91-133, August. 
Washington, DC: National Center for Education Statistics. 

Harris, J.S. (1992a), "Office Instructions for 1992 Teacher Follow-up Survey (TPS) 

Reinterview." Census Bureau TPS Office Memorandum No. 92-9, to All Regional 
Directors. 

(1992b), "Instructions for Conducting 1992 TPS Reinterview." Census Bureau 

TFS Office Memorandum No. 92-10, to All TFS Reinterviewers. 

Healy, P.E. (1990a), "Review of Completed SASS- 1 A Questionnaires from 1990 Pretest." 
Internal Census Bureau memorandum to Sarah A. Doherty, July 31. 

(1990b), "State Data Project - LEA Data Provided by Selected States in the 

1990 SASS Pretest." Internal Census Bureau memorandum for the record, Octet er 10. 

Healy, P.E., and Pasqualucci, L. (1991), "Observation Trip to Jeffersonville (March 11-12, 
1991)." Internal Census Bureau memorandum to Sarah A. Doherty, March 26. 

Holt, A., and Scanlon, B. (19Q4), "QED Conversion of the 1990-91 Schools and Staffing 
Survey (SASS) School and Administrator Records," forthcoming report to NCES. 



R-4 



ERIC 



J 67 



Hudson, L., and Darling-Hammond, L. (1987), "The Schooling and Staffing Surveys: 

Analysis Recommendations." RAND, draft report prepared for Center for Statistics, 
unpublished, 

Jackson, B. (1993), "Analysis of the 1991 Private School Survey (PSS) List Frame Adds." 
Internal Census Bureau draft memorandum for the record. 

Jackson, B., and Frazier, R. (1994), "Improving the Coverage of Private Elementary- 
Secondary Schools." Paper presented at the annual meeting of the American Statistical 
Association, section on Survey Research Methods, Toronto, Canada, August. 

Jefferson-Copcland, R. and Bynum, L. (1990), "1990 Field Test for the Schools and Staffing 
Survey — Review of the Public and Private School Administrator Questionnaires 
(SASS 2A/2B)." Internal Census Bureau memorandum for the record, May 31. 

Jenkins, C. (1992a), " A Review of the Post-edit Item Response Rates, Pre-edit Reject Rates, 
and Edit Change Tallies from the 1991 Schools and Staffing Survey." Internal Census 
Bureau memorandum for distribution list, March 23. 

(1992b), "Questionnaire Research in the Schools and Staffing Survey: A 

Cognitive Approach", Proceedings of the Section on Survey Research Methods, 
American Statistical Association, pp. 434-439. Alexandria, VA: American Statistical 
Association. 

Jenkins, C, Ciochetto, S., and Davis, W. (1992), "Results of Cognitive Research on the 

Public School 1991-92 Field Test Questionnaire for the Schools and Staffing Survey 
(SASS-3A)." Internal Census Bureau memorandum to distribution list, June 15. 

Jenkins, C. and DeMaio, T. (1990), "Report on Cognitive Research on the Public and Private 
School Questionnaires for the Schools Staffing Survey (SASSOA and SASS-3B)." 
Internal Census Bureau memorandum to distribution list, July 11. 

Jenkins, C and WetzcL A. (1994a), "The Results of the 1992 Teacher Follow-up Survey 
Reintcrview and Extensive Reconciliation: Part One of a Two-Part Report." Internal 
Census Bureau memorandum to distribution list, March 14. 

(1994b), "The 1992 Teacher Followup Survey and Extensive 

Reconciliation." Paper presented at the annual meeting of the American Statistical 
Association, section on Survey Research Methods, Toronto, Canada, August, 

Johnson, F. (1993), "Comparison of School Locale Settings: Self-Reported vs. Assigned." 
Proceedings of the Section on Survey Research Methods, American Statistical, 
Association, p. 686. Alexandria, VA: American Statistical Association, 



ERLC 



R-5 



lb 



Joseph, S, and OH veto T. (1992), "Plan for the Automated Data Collection Instrument 
Project. 11 Internal Census Bureau memorandum for the record, February 19, 

Kasprzyk, D., Salvucci, S., Saba, M, and Zhang, F. (1994), "Some Data Issues in School- 
Based Surveys," Paper presented at the annual meeting of the American Statistical 
Association, section on Survey Research Methods, Toronto, Canada, August. 

Kasprzyk, D., and Scheuren, F, (1994), "Intersurvey Consistency in School Surveys," Paper 
presented at the annual meeting of the American Statistical Association, section on 
Survey Research Methods, Toronto, Canada, August. 

Kaufman, S, (1988), "Schools and Staffing Pretest Methodology Analysis/ NCES Technical 
Report, CS 89-115, December. U,S. Department of Education, Office of Educational 
Research and Improvement, Washington, DC: National Center for Education Statistics, 

(1990), "Inconsistencies between School and Teacher File FTE Counts." 

Internal NCES memorandum to P. Planchon, January 25. 

(1991), "1988 Schools and Staffing Survey Sample Design and Estimation", 

Technical Report, NCES 91-127, May, U,S. Department of Education, Office of 
Educational Research and Improvement. Washington, DC: National Center for 
Education Statistics. 

(1992), "Balanced Half-sample Replication with Aggregation Units," 

Proceedings of the Section on Survey Research Methods, American Statistical 
Assoc iation, pp, 440-445. Alexandria, VA: American Statistical Association. 

(1993), "A Bootstrap Variance Estimator for the Schools and Staffing Survey," 

Proceedings of the Section on Survey Research Variance Methods, American Statistical 
Association, p. 675, Alexandria, VA: American Statistical Association. 

(1994), "Properties of the Schools and Staffing Survey Bootstrap Variance 

Estimator." Paper presented at the annual meeting of the American Statistical 
Association, section on Survey Research Methods, Toronto, Canada, August. 

Kaufman, S., and Huang, H. (1993), "1990-91 Schools and Staffing Survey: Sample Design 

and Estimation", Technical Report, NCES 93-449, July. U.S. Department of Education, 
Office of Educational Research and Improvement. Washington, DC: National Center 
for Education Statistics. 

Kindel, C. (1989), "Data Quality for SASS 1987-88." Internal NCES memorandum to Paul 
Planchon, February 9. 



R-6 

169 



(1992), "Electronic Data Collection at the National Center for Education Statistics: 

Successes in the Collection of Library Data," 1992 Annual Research Conference 
Proceedings, U.S. Department of Commerce. Washington, DC: U.S. Bureau of the 
Census. 

King, K., and Kaufman, S. (1994), "Estimation Issues Related to the Student Component of 
the Schools and Staffing Survey." Paper presented at the annual meeting of the 
American Statistical Association, section on Survey Research Methods. Toronto, 
Canada, August. 

Moonesinghe, R., Smith, W., and Gruber, K. (1993), "Characteristics of Nonrespondents to 
the 1990-91 Schools and Staffing Survey." Proceedings of the Section on Survey 
Research Methods, American Statistical Association, p. 698. Alexandria, VA: 
American Statistical Association. 

Nash, F. (1988), "SA^S School Districts that Refused for Schools Summary of Results." 
Internal NCES memorandum for the record, April 18. 

■ (n.d.), "Schools and Staffing Survey (SASS) Field Test Evaluation of Survey 

Forms." Internal NCES report. 

National Center for Education Statistics (1991a), "1987-88 Schools and Staffing Survey, 

Public and Private School Questionnaires, Base Year: Data File User's Manual." NCES 
91-139g. Washington, DC: National Center for Education Statistics. 

(1991b), "1987-88 Schools and Staffing Survey, 

Public and Private School Administrator Questionnaires, Base Year: Data File User's 
Manual." NCES 91-137g. Washington, DC: National Center for Education Statistics. 

(1991c), "1987-88 Schools and Staffing Survey, 

Public and Private Teacher Demand and Shortage Questionnaires, Base Year: Data File 
User's Manual." NCES 91-021g. Washington, DC: National Center for Education 
Statistics. 

(199 Id), "1987-88 Schools and Staffing Survey, 

Public and Private School Teacher Questionnaires, Base Year: Data File User's 
Manual." NCES 91-139g. Washington, DC: National Center for Education Statistics. 

(1992), "An Overview of the SASS and 'ITS." 

Washington, DC: National Center for Education Statistics. 

National Education Association (1990). "Rankings of the States, 1989." Washington, DC: 
author. 

Nelson, F.H. (1990), "Survey and Analysis of Survey Trends, 1989." p. 5.19. Washington, 
DC: American Federation of Teachers. 

R-7 



ERLC 



170 



Newbrough, J. (1989), "Report of SASS-2/3(R) and SASS-4(R) Reintcrview." Internal Census 
Bureau report, June 28. 

Nisselson, H., Parke, B., Streett, S., Salvucci, S., and Fink, S. (1993), "A Design for 

Assessing the Data Quality of the Common Core of Data (CCD)," Report submitted to 
NCES, September. 

Office of Educational Research and Improvement (n.d.), "National Center for Education 

Statistics Databases: The Schools and Staffing Survey." U.S. Department of Education, 
Office of Educational Research and Improvement. Washington, DC: Office of 
Educational Research and Improvement. 

Parmer, R.J., Shen, P., and Tan, A.I. (1992), "Mail vs. Telephone Response in the 1991 
Schools and Staffing Survey." Proceedings of the Section on Survey Research 
Methods, American Statistical Association, pp. 446-451. Alexandria, VA: American 
Statistical Association. 

Pasqualucci, L. (1991), "1991 Schools and Staffing Survey - Review of Field 

Representative's Edit and Performance Record (SASS-23)." Internal Census Bureau 
memorandum to Sarah A. Doherty, October 30. 

Peng, S., Gruber, K., Smith, W., and Jabine, T. (1993), "Monitoring Data Quality in 

Education Surveys," Proceedings of the International Conference on Establishment 
Surveys, p. 244. Alexandria, VA: American Statistical Association. 

Royce, D. (1990), "1989 Teacher Followup Survey (TFS) Reinterview Response Variance 
Report." Internal Census Bureau report, August 16. 

(1992), "1991 Schools and Staffing Survey (SASS) Reinterview Response Variance 

Report." Internal Census Bureau report, September 11. 

(1993), "The Teacher List Validation Study (TLVS): A Description", draft Census 

Bureau memorandum for the record, March 19. 

(1994), "Results from the 1993 Teacher List Validation Study (TLVS)." Internal 

Census Bureau report, Draft 2, March 17. 

Royce, D., and Schreiner, L (1994), "Improving Coverage in a National Survey of Teachers." 
Paper presented at the annual meeting of the American Statistical Association, section 
on Survey Research Methods, Toronto, Canada, August. 

Saba, M., Zhang, F., and Chang, M. (1994), "CCD Adjustment to the 1990-91 Schools and 
Staffing Survey (SASS)," forthcoming report to NCES. 



R-8 



171 



Salvucci, S., Galfond, G., and Kaufman, S. (1993), "Generalized Variance Functions for the 
Schools and Staffing Surveys," Proceedings of the Section on Survey Research 
Methods, American Statistical Association, p. 669. Alexandria, VA: American 
Statistical Association. 

Salvucci, S., and Holt, A. (1992), "Generalized Variance Estimates for 1987-88 Schools and 
Staffing Surveys," Unpublished, 

Salvucci, S., Holt, A., and Moonesinghe, R. (1994), "Generalized Variance Estimates for 
1990-91 Schools and Staffing Survey (SASS)," forthcoming report to NCES. 

Scheuren, F., Parke, R., and Bureika, R. (1994), "Schools and Staffing Survey (SASS) 1990- 
1991, Characteristics of Nonrespondents," forthcoming report to NCES. 

Schwanz, D. (1987), "1987-88 School and Staffing Surveys (SASS) Field Test: Comparison 
of Treatments' Effects on Teacher Response Rates." Internal NCES memorandum to 
C Kindel, August 14. 

Shen, P., Parmer, RJL, and Tan, A.I. (1992), "Characteristics of Nonrespondents in the 

Schools and Staffing Surveys' School Sample." Proceedings of the Section on Survey 
Research Methods, American Statistical Association, pp. 452-457. Alexandria, VA: 
American Statistical Association. 

Smith, W., and Ghosh, D. (1994), "Schools and Staffing Survey (SASS) Periodicity," 
forthcoming report to NCES. 

Smith, W., and Salvucci, S. (1989), "Supplemental Report on the Comparative Analysis of 
Private School Data Collections." Report submitted to NCES, August. 

Waite, P.J. (1990), "Weighting Specifications for the 1988-89 Teacher Follow-up Survey 
(TFS). ,f Internal Census Bureau memorandum to B. Gregory Russell, March 27. 

Whitener, S., Rohr, C, Bynum, L., Kaufman, S., and King, K. (1994), 1991-92 Teacher 

Followup Survey: Data File User's Manual. NCES 94-331 (Public Use Version) and 
NCES 94-478 (Restricted Use Version). U.S. Department of Education, Office of 
Educational Research and Improvement. Washington, DC: National Center for 
Education Statistics. 

Williams, J.W. (1992), "Results of the 1991 survey field test." Internal NCES memorandum to 
participants in school library media center statistics meeting, April 14. 

Wright, D. (n.d.), "Early Elementary and Secondary Redesign Research - The School and 
Staffing Survey." Internal NCES report. 



R-9 

172 



ISBN 0-16-045086-1 



United States 
Department of Education 
Washington, DC 20208-5651 

Official Business 
. Penalty for Private Use, $300 



Postage and-Fees Paid ■ 
U.S. Department of Education 
Permit No G-17 

Fourth Class |>pecial 
Special Handling 




